Distributed Applications with GO

Elements of a Distributed System

Characteristic

Four aspects might be

• Service Discovery
• Load Balancing
• Distributed tracing and logging
• Service Monitoring

Type of Distributed System

• Hub and Spoke (Satélite approach)
  • Advantages Good for load balancing and logging
  • Disadvantages Bad to single point of failure. Hub is complex due to responsibilities
• Peer to Peer where each communicate directly
  • Advantages No Single point of failure. Highly decoupled
  • Disadvantages Service discovery and Load Balancing hard
• Message Queue System where services get work from the queue
  • Advantages Easy to scale, Persistence for disaster
  • Disadvantages Single Point of failure (message queue), hard to configure
• Hybrid system (none of the above)
  • This might will have advantages and disadvantage of both

Architectural Element

These are the aspect you may want to consider

• Languages
• Frameworks (Recommended Go-Kit and Go-Micro)
• Transports
• Protocol

Sample App

The sample app is a hybrid app using GO

This is the components to build

Introduction

I do not usually go through large portions of code but I thought it might be useful to look at the sample code and comment on the topic and the relationship with GO as a language.

Project Structure

The project structure was basically a root folder with a cmd directory holding the main.go code for each binary. From there there is one folder for each component.

Log

Client

Comments for the code

• SetClientLogger - Sets the attribute for the standard log package
• Write - Writes data to server endpoint

The Code

package log

import (
	"app/registry"
	"bytes"
	"fmt"
	stlog "log"
	"net/http"
)

func SetClientLogger(serviceURL string, clientService registry.ServiceName) {
	stlog.SetPrefix(fmt.Sprintf("[%v] - ", clientService))
	stlog.SetFlags(0)
	stlog.SetOutput(&clientLogger{url: serviceURL})
}

type clientLogger struct {
	url string
}

func (cl clientLogger) Write(data []byte) (int, error) {
	b := bytes.NewBuffer([]byte(data))
	res, err := http.Post(cl.url+"/log", "text/plain", b)
	if err != nil {
		return 0, err
	}
	if res.StatusCode != http.StatusOK {
		return 0, fmt.Errorf("Failed to send log message. Service responded with %v - %v", res.StatusCode, res.Status)
	}
	return len(data), nil
}

Server

Comments for the code

This creates an instance of a custom log type, a handler and a function to write to the file.

• Run - Creates a custom log file using the standard log package
• Write - Writes data to the stream
• RegisterHandlers - Registers the "/log", reads the data and writes the message

The Code

package log

import (
	"io/ioutil"
	stlog "log"
	"net/http"
	"os"
)

var log *stlog.Logger

type fileLog string

func (fl fileLog) Write(data []byte) (int, error) {
	f, err := os.OpenFile(string(fl), os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0600)
	if err != nil {
		return 0, err
	}
	defer f.Close()
	return f.Write(data)
}

func Run(destination string) {
	log = stlog.New(fileLog(destination), "", stlog.LstdFlags)
}

func RegisterHandlers() {
	http.HandleFunc("/log", func(w http.ResponseWriter, r *http.Request) {
		msg, err := ioutil.ReadAll(r.Body)
		if err != nil || len(msg) == 0 {
			w.WriteHeader(http.StatusBadRequest)
			return
		}
		write(string(msg))
	})
}

func write(message string) {
	log.Printf("%v\n", message)
}

Service Registration

• Create Web Service
• Create Register Service
• Register Web Service
• Deregister Web Service

Service Discovery

• Create Grading Service
• Request Required Service On Startup
• Notify when Service Starts
• Notify when Service Shutdown