GO Web Services

Handling Requests

These are the two main ways to handle requests

• Handle a handler to handle requests matching a pattern
• HandleFunc a function to handle requests matching a pattern

Handle

Here is the signature for Handle.

func Handle(pattern string, handler Handler)

Handler is an interface which has the signature

type Handler interface {
	ServeHTTP(ResponseWriter, *Request)
}

So here is how to use this approach. We create a type and implement the interface required.

// Creating a handler type
type fooHandler struct {
  Message string
}

// Implementing the function for the interface
func (f *fooHandler) ServeHTTP(w http.ResponseWrite, r *http.Request) {
  w.Write( []byte(f.Message) )
}


func main() {
  http.Handle("/foo", &fooHandler{Message: "hello Word"})
}

Remember Remember we can construct a struct using a struct literal e.g.

type Student struct {
    Name string
    Age  int
}

pb := &Student{ // pb == &Student{"Bob", 8}
    Name: "Bob",
    Age:  8,
}

Requests

These consist of three parts

• Method
• Headers
• Body

HandleFunc

Perhaps somewhat simpler. We just need to create a function with the correct signature

func main() {
  foo := funct(w http.ResponseWritter, _ *http.Response) {
    w.Write( []byte("hello world") )
  }

  http.HandleFunc("/foo", foo)
}

Handling JSON

Marshal and Unmarshal

Every GO type implements the empty interface so any struct can be marshaled into JSON.

func Marshal(v interface{}) (byte[]. error)

To ensure the struct is marshaled, all the fields need to be exported. In this example surname would not be marchaled.

type foo struct{
  Message string
  Age int
  surname string
}

json,Marshal(&foo{"4Score",56, "Lincoln"})

To Unmarshal

f := foo{}
json.Unmarshal([]byte(`{"Message":"4Score","Age":56, "Name":"Abe"}`), &f)

Demo Data

Demo Type

Lets make a type to encode

package product

// Product
type Product struct {
	ProductID      int    `json:"productId"`
	Manufacturer   string `json:"manufacturer"`
	Sku            string `json:"sku"`
	Upc            string `json:"upc"`
	PricePerUnit   string `json:"pricePerUnit"`
	QuantityOnHand int    `json:"quantityOnHand"`
	ProductName    string `json:"productName"`
}

Make Some Demo Data

Here is some demo data.

[
  {
    "productId": 1,
    "manufacturer": "Johns-Jenkins",
    "sku": "p5z343vdS",
    "upc": "939581000000",
    "pricePerUnit": "497.45",
    "quantityOnHand": 9703,
    "productName": "sticky note"
  },
  {
    "productId": 2,
    "manufacturer": "Hessel, Schimmel and Feeney",
    "sku": "i7v300kmx",
    "upc": "740979000000",
    "pricePerUnit": "282.29",
    "quantityOnHand": 9217,
    "productName": "leg warmers"
  },
...

Below is a example of how to read the json into a map

func loadProductMap() (map[int]Product, error) {
	fileName := "products.json"
	_, err := os.Stat(fileName)
	if os.IsNotExist(err) {
		return nil, fmt.Errorf("file [%s] does not exist", fileName)
	}

	file, _ := ioutil.ReadFile(fileName)
	productList := make([]Product, 0)
	err = json.Unmarshal([]byte(file), &productList)
	if err != nil {
		log.Fatal(err)
	}
	prodMap := make(map[int]Product)
	for i := 0; i < len(productList); i++ {
		prodMap[productList[i].ProductID] = productList[i]
	}
	return prodMap, nil
}

Implementing Method Types

Create Handler and Determine Mathod

We can now provide a GET method within the handler. The handler determines which method by looking at the request.

func handleProduct(w http.ResponseWriter, r *http.Request) {
...
	switch r.Method {
            // GET
            // POST
            // etc
        }
}

GET Method

We use the Mnmarshal method to encode the json

	case http.MethodGet:
		productsJSON, err := json.Marshal(productList)
		if err != nil {
			w.WriteHeader(http.StatusInternalServerError)
			return
		}
		w.Header().Set("Content-Type", "application/json")
		w.Write(productsJSON)

POST Method

We use the Unmarshal method to decode the json. In the source they replaced the ReadAll Unmarshal in favour of json.NewDocder.

	case http.MethodPost:
		// add a new product to the list

		var newProduct Product

		bodyBytes, err := ioutil.ReadAll(r.Body)
		if err != nil {
			w.WriteHeader(http.StatusBadRequest)
			return
		}
		err = json.Unmarshal(bodyBytes, &newProduct)
		if err != nil {
			w.WriteHeader(http.StatusBadRequest)
			return
		}

		err := json.NewDecoder(r.Body).Decode(&product)
		if err != nil {
			log.Print(err)
			w.WriteHeader(http.StatusBadRequest)
			return
		}


		if newProduct.ProductID != 0 {
			w.WriteHeader(http.StatusBadRequest)
			return
		}
		newProduct.ProductID = getNextID()
		productList = append(productList, newProduct)
		w.WriteHeader(http.StatusCreated)

Managing the Routes

Example Routes

With REST we generally have one route for all products and one for a particular product. e.g. GET /products/123. To manage this we greate different handlers.

	http.Handle("/products", middlewareHandler(productListHandler))
	http.Handle("/products/", middlewareHandler(productItemHandler))

Managing the URL

Here is an example of getting the 123 from the URL. Note we return a 404 if we cannot determine the resource and not a bad request.

func productHandler(w http.ResponseWriter, r *http.Request) {
	urlPathSegments := strings.Split(r.URL.Path, "products/")
	productID, err := strconv.Atoi(urlPathSegments[len(urlPathSegments)-1])
	if err != nil {
		log.Print(err)
		w.WriteHeader(http.StatusNotFound)
		return
	}
	product, listItemIndex := findProductByID(productID)
	if product == nil {
		http.Error(w, fmt.Sprintf("no product with id %d", productID), http.StatusNotFound)
		return
	}
	switch r.Method {
	case http.MethodGet:
...

Middeware

Explanation

I liked this example provided in GO. Just made sense to me. This is like the interceptors in Angular.

func enableCorsMiddleware(handler http.Handler) http.Handler {

  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

  // do stuff before intended handler here 

  handler.ServeHttpp(w,r)

  // do stuff after intended handler here 

})

func intendedFunction(w http.ResponseWriter, r *http.Request) {
  // business logic here
}

func main() {
  intendedHandler := http.HandlerFunc(intendedFunction)
  http.Handle("/foo", enableCorsMiddleware(intendedHandler))
  http.ListenAndServe(":5000", nil)
}

Wrapping Handlers

When using middleware we need to convert our handler to http.handlers. To do this we use the http.handlerFunc. E.g.

// Before Middleware

//	http.HandleFunc("/products", productsHandler)
//	http.HandleFunc("/products/", productHandler)
//	http.ListenAndServe(":5000", nil)

func main() {
	productListHandler := http.HandlerFunc(productsHandler)
	productItemHandler := http.HandlerFunc(productHandler)

	http.Handle("/products", middlewareHandler(productListHandler))
	http.Handle("/products/", middlewareHandler(productItemHandler))
	http.ListenAndServe(":5000", nil)
}

Yet another CORs

Well love diagrams to here are another two. CORs prevents same-site access. Where means different ports, protocols subdomains are not allowed.


Below is a sample function to enable CORs. Do it, do it, don't do it.

func enableCorsMiddleware(handler http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Add("Access-Control-Allow-Origin", "*")
		w.Header().Add("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
		w.Header().Add("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Authorization, X-CSRF-Token, Accept-Encoding")
		handler.ServeHTTP(w, r)
	})
}

Persisting Data

Connection to Database

Nothing too difficult here.

package services

import (
  "database/sql"
  "strconv"
  "log"
)

type DatabaseConnection struct {
  Db *sql.DB
}

func NewDatabaseConnection(
  dbHost string,
  dbPort int,
  dbName, dbUser, dbPass string) *DatabaseConnection {

  var connectionString = 
    dbUser + ":" + dbPass + "@tcp(" + dbHost + ":" + strconv.Itoa(dbPort) + ")/" + 
    dbName + "?parseTime=true&autocommit=false"

   dbConnection, err := sql.Open("mysql", connectionString)
   if err != nil {
    log.Fatal("Error connecting to Database. Error is %s", err.Error())
   }
}

Doing a SELECT

We create a Select Statement using prepare to do this once as it is expenseive

func CreateSelectStatement(databaseConnection *DatabaseConnection) *sql.Stmt {

  selectStatement, err := databaseConnection.Db.Prepare(
   "SELECT NEW_CASES, NEW_DEATHS, NEW_RECOVERED, TOTAL_CASES, TOTAL_DEATHS, TOTAL_RECOVERED, RECORD_DATE_TIME FROM COVID_DATA WHERE ALPHA2CODE = ? AND YEAR = ? AND DAY = ?")

  if err != nil {
     log.Fatal("Error Creating Select Statement. Error was %s", err.Error())
  }

  return selectStatement
}

Now we can bind the arguments to the statement

func (covidDataTable *CovidDataTable) Select(
  alpha2Code string,
  year, day int) (bool, *models.CovidData) {

  // Get the rows
  rows, err := covidDataTable.selectStatement.Query(&alpha2Code, &year, &day)
  if err != nil {
    log.Fatal("Error occurred selecting covid data. Error was  %s", err.Error())
    return false, nil
  }
...

And then we can iterate over the results and make sure we close the rows if there is an error.

...

defer rows.Close()

  ok := rows.Next()
  if !ok {
     log.Fatal("Error occurred Selecting covid data for Country %s, Year %d, Day.", alpha2Code, year, day)
  }

  var newConfirmed, newDeaths, newRecovered int
  var totalConfirmed, totalDeaths, totalRecovered int
  var recordDateTime time.Time

  err = rows.Scan(
    &newConfirmed, &newDeaths, &newRecovered,
    &totalConfirmed, &totalDeaths, &totalRecovered, &recordDateTime)

    if err != nil {
      log.Fatal("Error occurred selecting covid data. Error was  %s", err.Error())
                return false, nil
    }
...

Doing a INSERT

We create a Insert Statement using prepare to do this once as it is expenseive

func CreateInsertStatement(databaseConnection *DatabaseConnection) *sql.Stmt {

  insertStatement, err := databaseConnection.Db.Prepare(
    "INSERT INTO COVID_DATA(ALPHA2CODE, YEAR, DAY, NEW_CASES, NEW_DEATHS, NEW_RECOVERED, TOTAL_CASES, TOTAL_DEATHS, TOTAL_RECOVERED, RECORD_DATE_TIME) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)")

  if err != nil {
     log.Fatal("Error Creating Insert Statement. Error was %s", err.Error())
  }
  return insertStatement
}

And now we can to the insert

func (covidDataTable *CovidDataTable) Insert(
  tx *sql.Tx,
  alpha2Code string,
  year, day int,
  covidData *models.CovidData) bool {

  // Do the insert
  result, err := tx.Stmt(covidDataTable.insertStatement).Exec(
    &alpha2Code,
    &year,
    &day,
    covidData.NewConfirmed,
    covidData.NewDeaths,
    covidData.NewRecovered,
    covidData.TotalConfirmed,
    covidData.TotalDeaths,
    covidData.TotalRecovered,
    covidData.RecordDateTime)

    var myError = covidDataTable.insertStatement.Close()
    if myError != nil {
      log.Fatal("Yes my son %s", myError.Error())
    }

    // Check for Error
    if err != nil {
      log.Fatal("Error occurred inserting covid data. Error Executing statement. Error was  %s", err.Error())
    }

    rows, err := result.RowsAffected()
    if err != nil {
      logFatal("Error occurred inserting covid data.  Error getting row affected. Error was  %s", err.Error())
    }

    // All good
    return true
}

Managing Database Resources

Connection Pools

These are the parameters available.

• Connection Max Lifetime - Sets maximum amount of time a connection may be used
• Max Idle Connections - Sets the maxium number in idle connection pool - default is 2
• Max Open Connections - Sets the maximum open connections

Using Contexts

These allow you to perform tasks with timeouts. Useful to ensure resource are no exhausted.

func getProduct(productID int) (*Product, error) {
	ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
	defer cancel()
	row := database.DbConn.QueryRowContext(ctx, `SELECT 
	productId, 
	manufacturer, 
	sku, 
	upc, 
	pricePerUnit, 
	quantityOnHand, 
	productName 
	FROM products 
	WHERE productId = ?`, productID)

	product := &Product{}
	err := row.Scan(
		&product.ProductID,
		&product.Manufacturer,
		&product.Sku,
		&product.Upc,
		&product.PricePerUnit,
		&product.QuantityOnHand,
		&product.ProductName,
	)

	if err == sql.ErrNoRows {
		return nil, nil
	} else if err != nil {
		log.Println(err)
		return nil, err
	}
	return product, nil
}

File Uploading

To upload files we could

• encode the file to base64 and convert to Json
• multipart/form-data type which uses HTTP form submit

Encode

To decode the string we would need to

str := "SGVsbG8gV29ybGQ="

output, err := base64.StdEncoding.DecodString(str)
if err != nill {
  log.Fatal(err)
}

fmt.Printf("%q\n", output)

Using Multipart

Uploading

The ParsMultiPartForm will start using disk if the limit is exceeded.
For the roll left bit

op1 << op2 Shift bits of op1 left by distance op2; fills with zero bits on the right-hand side
5    << 20
0101 << 00000000000000000000
010100000000000000000000 (binary) = 50000 (Bytes)

func uploadFileHandler(w http.ResponseWriter, r *http.Request) {
  r.ParseMultipartForm(5 << 20) // 5 Mb
  file, handler, err := r.FormFile("receipt")
  if err != nil {
    log.Print(err)
    w.WriteHeader(http.StatusBadRequest)
    return
  }

  defer file.Close()

  f, err := os.OpenFile(filepath.Join(ReceiptDirectory, handler.Filename), os.O_WRONLY|os.O_CREATE, 0666)
  
   defer f.Close()
   io.Copy(f, file)
}

Downloading

And here is an example of Downloading

  fileName := urlPathSegments[1:][0]
  file, err := os.Open(filepath.Join(ReceiptDirectory, fileName))
  defer file.Close()
  if err != nil {
	w.WriteHeader(http.StatusNotFound)
	return
  }
  
  fHeader := make([]byte, 512)
  file.Read(fHeader)
  fContentType := http.DetectContentType(fHeader)

  stat, err := file.Stat()
  if err != nil {
	w.WriteHeader(http.StatusInternalServerError)
	return
  }

  fSize := strconv.FormatInt(stat.Size(), 10)
  w.Header().Set("Content-Disposition", "attachment; filename="+fileName)
  w.Header().Set("Content-Type", fContentType)
  w.Header().Set("Content-Length", fSize)
  file.Seek(0, 0)
  io.Copy(w, file)