TABLE OF CONTENTS

TL;DR

This article is for those who want to quickly glance over Golang and build a small project, it serves as an introduction into the language.

After going through the post you will know how to build a simple CRUD app and you will be somehow familiar with Go syntax to start your own project.

If you have feedback you can reach me out on Twitter @tekbog or leave a comment here.

The repo for this project can be found HERE.

If you are unfamiliar with API/backend development I have starting series with both Node and Spring Boot you can check.

Why Golang?

If you want to build loosely coupled micro services that are low on memory and scalable Go is probably one of the best choices, if not the best, considering the fact that Kubernetes, Docker and Terraform.

Additionally many other companies use it such as Uber and of course the tech giant Google is using it heavily internally.

Overall Go excels in the back end with Goroutines and Channels. It offers quick development with a minimalist C-like language.

Go Syntax

You can use this page as the Go syntax cheat sheet or start the Go tour from the official site.

Concurrency

I'm not going to go into concurrency, however here's a good enough tutorial to get you started.
To paraphrase a bit:

Goroutine is a lightweight thread.

Channel allows to send data between Goroutines.

Now keep in mind that while looking for Go docs the last release as of today 1.18 has introduced Generics so some of the old tutorials might be outdated.

Let's get through some basics to get you started.

Imports and exports

In every new file you will need to declare its package right at the start such as: package services

Then in order to use export (or expose) a function you need to start it with a capital letter. In the example above you can see that the function GetFeedTweets starts with a capital letter, this means I'm making it "public".
Afterwards just import the package like: import ("project-folder-name/package") and you will be use the function by invoking the package name followed by the dot and the function like: package.FunctionName.

Note that when working with Structs you have to make the fields start with capital letter if you want to expose them. Otherwise you will end like me:

Golang is case sensitive

REMEMBER THIS OR YOU WILL SUFFER

Pointers

I won't be covering pointers in detail as it's a lengthy topic, however here's an article that should make a good introduction.

The Go tour also offers a quick introduction.

To sum it up (badly, please read up on it):

COPY

//declare int variable that equals to 42
var i = 42
//make p an int pointer
var p *int
//assign the i memory address to p
p = &i

//p is just a pointer holding memory
//will print 0xc0000b8000
fmt.Println(p)

//p* points to the memory address it's holding
//will print 42
fmt.Println(*p)

You will see functions getting parameters like func FuncName (p *pointerType) just use p as you normally would.

Variadic functions:

If you come from the JavaScript world you will know how useful this is as it works the same as the spread operator, here's a quick tutorial.

Here's a quick example from StackOverflow:

COPY

package main
import "fmt"

func my_func( args ...int) int {
   sum := 0
  //the underscore in golang is called the "blank identifier"
   for _, v := range args {
      sum = sum + v
   }

   return sum;
}

func main() {
    arr := []int{2,4}
    sum := my_func(arr...)
    //prints: Sum is  6
    fmt.Println("Sum is ", sum)
}

However it's easier to understand how it works with strings:

COPY

package main

import (
    "fmt"
    "strings"
)

func joinString(element ...string) string {
    return strings.Join(element, ",")
}

func main() {

    //will print "join,this"
    fmt.Println(joinString("join", "this"))
    //will print "a,a,a"
    fmt.Println(joinString("a", "a", "a"))

}

Make

The quickest way to make a slice, which is a dynamic array, just do

COPY

a := make([]int, 5)

Here's the page from the Go tour

Range

Golang doesn't have a forEach function however it uses range instead so if you want to loop through all the elements of an array or slice just do:

COPY

for index, element := range yourSlice{
        fmt.Println(index)
        fmt.Println(element)
}

You can find more examples in this site with golang code examples.

Maps

Maps, hash tables or dictionaries - in Golang they are used as follows:

COPY

    m := map[string]int{
        "first-key": 5,
        "second-key":  10,
    }
    fmt.Println(m) 

    for key, value := range m { // Order not specified
        fmt.Println(key, value)
    }

Here's a quick summary about maps.

Methods:

Golang doesn't have classes however it has methods. It sounds counter intuitive but it works great, if you have worked with OOP before just think of Structs as classes/objects. You define a function for them and then use that function on the Struct with a dot call, here's an example.

COPY

import "fmt"

type User struct {
    Id   int
    Name string
}

//method to set user's name
func (u *User) setName(newName string) {
    (*u).Name = newName
}

func main() {

    //initialize a new user
    newUser := User{
        Id:   0,
        Name: "First Name",
    }
    //User's ID:  0
    fmt.Println("User's ID: ", newUser.Id)
    //User's Name:  First Name
    fmt.Println("User's Name: ", newUser.Name)

    // make a pointer to the newUser
    pointerNewUser := &newUser

    // call your method
    pointerNewUser.setName("Name From Pointer")
    //User's Name:  First Name
    fmt.Println("Same User's Id: ", pointerNewUser.Id)
    //User's new name: Name From Pointer
    fmt.Println("User's new name:", pointerNewUser.Name)
}

Print

When you want to print something to the console remember to use the proper verbs:

COPY

%v    the value in a default format when printing structs, the plus flag (%+v) adds field names
%#v    a Go-syntax representation of the value
%T    a Go-syntax representation of the type of the value
%%    a literal percent sign; consumes no value

That's from the official documentation.

For example a normal expression would be:

COPY

fmt.Printf("checking user: %v\n", user)

TypeOf: Reflection

If you want to examine types during runtime then you need to use the reflect package:

COPY

package main

import (
    "fmt"
    "reflect"
)

type user struct {
    Id   int
    Name string
}

type blog struct {
    User_id int
    Data    string
    Date    string
}

//this is a generic function since I'm using the keyword "any"
func checkTypeAndValue(x any) {
    userType := reflect.TypeOf(x)
    userValue := reflect.ValueOf(x)
    fmt.Println("Type ", userType)
    fmt.Println("Value ", userValue)

}
func main() {
    newUser := user{
        Id:   1,
        Name: "myUser",
    }
    checkTypeAndValue(newUser)

    newBlog := blog{
        User_id: 1,
        Data:    "this is my first blog",
        Date:    "01-01-1990",
    }
    checkTypeAndValue(newBlog)

}

Blank identifier

You might find blank identifiers in loops or when you import a package, like:

COPY

  //the underscore in golang is called the "blank identifier"
   for _, v := range args {
      sum = sum + v
   }

You can find a good explanation from Effective Go about it. However to put it simple: the compiler complains if you aren't using a variable, so if you need to store something but never use it then use the underscore.

Generics

Finally here! The type of jokes no longer work:

With 1.18 Golang has introduced generics, so be careful when you look at old tutorials.

I'd recommend you to read on Generics if you haven't heard the term before but the gist of it is that you can use one function for many types of data, just like I did in the example of TypeOf: Reflection, there's a generic function there that accepts any type of data:

COPY

//this is a generic function since I'm using the keyword "any"
func checkTypeAndValue(x any) {
    userType := reflect.TypeOf(x)
    userValue := reflect.ValueOf(x)
    fmt.Println("Type ", userType)
    fmt.Println("Value ", userValue)

}

This FreeCodeCamp article makes it quick to understand. And you can also refer to the official Golang docs.

The project: mini Twitter backend

Notes before you start

Keep in mind this project was made mainly to show off Golang a bit, you can copy the architecture but the project is missing proper ORM, there's no authentication or authorization in place, I'm completely disregarding middleware and there are no tests.

I will talk about all this issues in its own sections but it's important that you are aware that this is not ready for production.

If I had to start from scratch or remake the project I'd add libraries such as sqlx and Gorm. As well as improve the API and other changes that I go through below.

Additionally I'd like to address the routing library I'm using: Fiber. Note that it's version 2, however a lot of tutorials and blog posts are showing and talking about v1, a bit has changed since then so when looking for additional information make sure the import is github.com/gofiber/fiber/v2.

Also, you should definitely have a configuration file and an .env in your project to hide your data.

There's also a disclaimer in the README:

Disclaimer

This is an introductory project to look at Golang a bit. This project is not production ready and it has bad practices like the way pagination works or how we interact with the database.

Through the codebase you will find different "prints" that are used to debug the project, feel free to play with them. What's missing for production ready?

You should add a proper logger, configuration, middleware, a different way to handle the data, perhaps an ORM, and a better way to handle the pagination, as well as a better API.

Note that you can make yourself vulnerable to SQL ingections if you copy and paste the code. This has been made for learning purposes.

Database

You should probably have Docker running MySQL (or any other SQL database). If not you can always install MySQL on your system and use something like MySQL Workbench to work with it.

Database design

I'm using QDBD for the diagram.

The database itself is fairly simple, you have the users that tweet tweets and a table of followers to keep the data of who follows who. The followers table is mainly to implement the feed/timeline for our users.

Create Database

In the scripts folder you will find the main script to run to start your MySQL database:

COPY

use twitterdb;

DROP TABLE IF EXISTS tweets;
DROP TABLE IF EXISTS users;
DROP TABLE IF EXISTS followers;

CREATE TABLE users (
  user_id INT NOT NULL AUTO_INCREMENT,
  user VARCHAR(255) NOT NULL,
  passhash VARCHAR(40) NOT NULL,
  email VARCHAR(255) NOT NULL,
  first_name VARCHAR(255) NOT NULL,
  last_name VARCHAR(255) NOT NULL,
  dob DATE,
  PRIMARY KEY (user_id)
);

CREATE TABLE tweets (
  tweet_id INT NOT NULL AUTO_INCREMENT,
  user_id INT NOT NULL,
  tweet VARCHAR(140) NOT NULL,
  date_tweet DATETIME NOT NULL,
  PRIMARY KEY (tweet_id),
  FOREIGN KEY user_id(user_id) REFERENCES users(user_id) 
  ON UPDATE CASCADE ON DELETE CASCADE
);

CREATE TABLE followers (
  id_user INT NOT NULL REFERENCES users (user_id),
  id_follower INT NOT NULL REFERENCES users (user_id),
  PRIMARY KEY (id_user, id_follower)
);

INSERT INTO users (user, passhash, email, first_name, last_name, dob) VALUES
("foo", "asdsad1", "test@gmail.com", "bob", "bobbinson", "2006-01-01"),
("foo2", "asdsad2", "test2@gmail.com", "bob2", "bobbinson2", "1992-01-01"),
("foo3", "asdsad3", "test3@gmail.com", "bob3", "bobbinson3", "1993-01-01"),
("foo4", "asdsad4", "test4@gmail.com", "bob4", "bobbinson4", "1994-01-01"),
("foo5", "asdsad5", "test5@gmail.com", "bob5", "bobbinson5", "1995-01-01"),
("foo6", "asdsad6", "test6@gmail.com", "bob6", "bobbinson6", "1996-01-01"),
("foo7", "asdsad7", "test7@gmail.com", "bob7", "bobbinson7", "1925-01-01"),
("foo8", "asdsad8", "test8@gmail.com", "bob8", "bobbinson8", "1980-01-01"),
("foo9", "asdsad9", "test9@gmail.com", "bob9", "bobbinson9", "1980-01-01"),
("foo10", "asdsad10", "test10@gmail.com", "bob10", "bobbinson10", "1970-01-01");

INSERT INTO tweets(user_id, tweet, date_tweet) VALUES
(1, "test tweet", "2001-01-01 22:00:00"),
(2, "test tweet2", "2002-01-01 22:00:00"),
(3, "test tweet3", "2003-01-01 22:00:00"),
(4, "test tweet4", "2004-01-01 22:00:00"),
(5, "test tweet5", "2005-01-01 22:00:00");

INSERT INTO followers(id_user, id_follower) VALUES
(5,1),
(4,1),
(3,1),
(2,1),
(6,1),
(2,5),
(4,5);

The other files are examples of queries that we will be using later.

Project Architecture

If you are familiar with building software this should be familiar if not, first let me tell you what's missing and afterwards we will go through every folder.

The project is missing folders like config, middleware, logger and testing. And if you would organize the controller better you'd have a routes folder where you organize the API better.

Quick note on imports

In this project you might get confused by imports such as:

COPY

import (
    "goexample/database"
    "goexample/models"
    "goexample/services/utils"
)

The goexample here is the name of the project, I just renamed my repo afterwards so it would make sense, so instead of having "mini-twitter-clone/database" we use "goexample/database" to import the database package.
With any other new projects just use the name of the folder.

API

You store your endpoints/routes in controller.go:

COPY

package api

import (
    "goexample/services"

    "github.com/gofiber/fiber/v2"
)

func SetupRoutes(app *fiber.App) {

    api := app.Group("/api")

    //get all unordened users
    api.Get("/users", services.GetUsers)
    //get all users ordered by age ASC
    api.Get("/users/age", services.GetUsersByAgeAsc)

    //get all unordened tweets from db
    api.Get("/tweets", services.GetTweets)

    //http://localhost:3000/api/feed/1
    //get MOST RECENT feed/timeline for the user
    api.Get("/feed/:id", services.GetFeedTweets)
    //pagination
    api.Get("/feed/:id/:limit/:offset", services.GetFeedTweetsPaginated)
    //can try https://github.com/gofiber/fiber/issues/193#issuecomment-591976894
    //a whole presentation on why you shouldn't do what I did:
    //https://use-the-index-luke.com/no-offset

}

I'm using the Fiber library, which is similar to Express, here we have several endpoints that we call with a Get request and once the server gets a request its response is to call the functions that I have exposed from the services package - that's where our business logic is.

NB: pagination isn't properly implemented.

This is just a quick example, this endpoint has security issues just like others but you should understand how the library and Golang works through the example.

Note that in the /feed/:id the id parameter pertains to the user we want to get the feed for.

Start the project

You can start the project by doing go run . inside the folder.
Your terminal should look like this:

Let's visit the different endpoints to see the responses, I'm going to use a normal browse but you should check Postman if you are not familiar with debugging the backend.

Let's run the endpoints from the controller:

COPY

http://127.0.0.1:3000/api/users

I left a lot of prints so if you check your terminal with every call you should see stuff like:

Next is a call where we order users by age:

COPY

http://127.0.0.1:3000/api/users/age

When we ask for tweets we get all of them:

COPY

http://127.0.0.1:3000/api/tweets

And now we get the feed or timeline for the user with the id 1:

COPY

http://127.0.0.1:3000/api/feed/1

Remember that pagination isn't production ready but the core concepts are the same:

COPY

http://127.0.0.1:3000/api/feed/1/2/1

All the calls work thanks to the business logic we have in the services package and the functions we have exposed.

Services

Inside the services package we have the functions that are called when someone or something hits one of the endpoints. Remember that we expose the functions by using a capital letter in the package.

Let's look at timeline_tweets.go which contains two functions for two different endpoints in the controller.go file:

COPY

package services

import (
    "fmt"
    "goexample/database"
    "goexample/models"
    "goexample/services/utils"
    "log"

    "github.com/gofiber/fiber/v2"
)

func GetFeedTweets(c *fiber.Ctx) error {

    //you shouldn't do this by the way, but it's just a demo
    // dbQuery := fmt.Sprintf("SELECT users.user_id, users.user, users.first_name, users.last_name, tweets.tweet, tweets.date_tweet FROM users INNER JOIN tweets ON users.user_id = tweets.user_id INNER JOIN followers ON users.user_id = followers.id_user WHERE followers.id_follower = %s ORDER BY tweets.date_tweet DESC;", c.Params("id"))
    // rows, err := database.DB.Query(dbQuery)

    //avoid the SQL injection by rewriting it like
    dbQuery := "SELECT users.user_id, users.user, users.first_name, users.last_name, tweets.tweet, tweets.date_tweet FROM users INNER JOIN tweets ON users.user_id = tweets.user_id INNER JOIN followers ON users.user_id = followers.id_user WHERE followers.id_follower = ? ORDER BY tweets.date_tweet DESC;"
    rows, err := database.DB.Query(dbQuery, c.Params("id"))

    //check for errors
    if err != nil {
        return utils.DefaultErrorHandler(c, err)
    }
    //close db connection
    defer rows.Close()

    //create a slice of tweets
    var timelineTweets []models.TimelineTweet
    //loop through the result set
    for rows.Next() {
        timelineTweet := models.TimelineTweet{}
        err := rows.Scan(&timelineTweet.User_id, &timelineTweet.User, &timelineTweet.First_name, &timelineTweet.Last_name, &timelineTweet.Tweet, &timelineTweet.Date_tweet)
        if err != nil {
            log.Fatal(err)
        }
        timelineTweets = append(timelineTweets, timelineTweet)
    }
    fmt.Print(timelineTweets)

    utils.ResponseHelperJSON(c, timelineTweets, "timeline", "No timeline found")

    return err
}

func GetFeedTweetsPaginated(c *fiber.Ctx) error {

    // dbQuery := fmt.Sprintf("SELECT users.user_id, users.user, users.first_name, users.last_name, tweets.tweet, tweets.date_tweet FROM users INNER JOIN tweets ON users.user_id = tweets.user_id INNER JOIN followers ON users.user_id = followers.id_user WHERE followers.id_follower = %s ORDER BY tweets.date_tweet DESC LIMIT %s OFFSET %s;", c.Params("id"), c.Params("limit"), c.Params("offset"))
    // avoid a SQL injection by rewriting it like
    dbQuery := "SELECT users.user_id, users.user, users.first_name, users.last_name, tweets.tweet, tweets.date_tweet FROM users INNER JOIN tweets ON users.user_id = tweets.user_id INNER JOIN followers ON users.user_id = followers.id_user WHERE followers.id_follower = ? ORDER BY tweets.date_tweet DESC LIMIT ? OFFSET ?;"

    rows, err := database.DB.Query(dbQuery, c.Params("id"), c.Params("limit"), c.Params("offset"))
    if err != nil {
        return utils.DefaultErrorHandler(c, err)
    }

    defer rows.Close()

    var timelineTweets []models.TimelineTweet
    for rows.Next() {
        timelineTweet := models.TimelineTweet{}
        err := rows.Scan(&timelineTweet.User_id, &timelineTweet.User, &timelineTweet.First_name, &timelineTweet.Last_name, &timelineTweet.Tweet, &timelineTweet.Date_tweet)
        if err != nil {
            log.Fatal(err)
        }
        timelineTweets = append(timelineTweets, timelineTweet)
    }
    //TODO: implement a response with pages and all that pagination jazz
    utils.ResponseHelperJSON(c, timelineTweets, "timeline", "No timeline found")

    return err
}

The first that you can notice is how we pass the context to GetFeedTweets and afterwards we use the variable 'c' to use that context as the Fiber library says.

Afterwards we use the exposed variable DB from the package database to open the DB, read the data and close it afterwards.

In order to store and scan the data properly we use a struct from our models package.

Afterwards you will see several functions from the utils package. Those functions are mainly helpers so you can check out how Golang does loops and certain other things.

The two other files inside our services package are very similar, our work here is get the data from the database, scan it through our array of structs and send it back to the user in JSON format. As well as we run some functions from the utils package.

Utils

This package contains helper functions, perhaps the most interesting being inside user_helper as it has functions that interact with data from slices, however be careful, they are not as well implemented as you might think.

Let's look at the most helpful one, response_helper.go:

COPY

package utils

import (
    "github.com/gofiber/fiber/v2"
)

//response JSON for services after you loop and scan
func ResponseHelperJSON(c *fiber.Ctx, data any, dataType string, dataError string) {
    if data != nil {
        c.Status(200).JSON(&fiber.Map{
            "success": true,
            dataType:  data,
        })
    } else {
        c.Status(404).JSON(&fiber.Map{
            "success": false,
            "error":   dataError,
        })
    }
}

Note that this functions contains a generic type of data as I'm using the keyword any and I'm interacting with the context given from Fiber through the variable 'c'.

Models

The models folder contains the different structs we use as entities to interact with our database as shown in the services folder.

Here's an example, note that even that if you want to expose the entire struct everything has to start with capital letter:

COPY

package models

type UserWithAge struct{
    Id         int   `json:"id"`
    User       string `json:"user"`
    Passhash   string `json:"passhash"`
    Email      string `json:"email"`
    First_name string `json:"first_name"`
    Last_name  string `json:"last_name"`
    Age        int    `json:"age"`
}

Database

The database package is very straightforward and remember to use a config file and an .env to store your sensitive data.

COPY

package database

import (
    "database/sql"
    "fmt"
    "log"
)

var DB *sql.DB

func Connect() error{
    var err error
    //use a config file for this
    DB, err = sql.Open("mysql", "root:password@tcp(127.0.0.1:3306)/twitterdb")

    if err != nil {
        log.Fatal(err)
        return err
    }

    if err = DB.Ping(); err != nil {
        log.Fatal(err)
        return err
    }

    fmt.Println("Connected to database")

    return nil

}

Main.go

Finally, this is where we start our server:

COPY

package main

import (
    "github.com/gofiber/fiber/v2"
    "goexample/database"
    "log"

    "goexample/api"

    _ "github.com/go-sql-driver/mysql"
)

func main() {

    if err := database.Connect(); err != nil {
        log.Fatal(err)
    }

    app := fiber.New()
    api.SetupRoutes(app)


    log.Fatal(app.Listen(":3000"))

}