Building a nice Postgres / Golang / HTTP Stack

Posted on by natebosscher

I’ve been struggling to find a nice set of frameworks for my day-to-day work. Yes, I’ve got those big packages like Golang, React, React Native, Postgres, TimescaleDB … but I’m talking about the finer things. Like how to setup your server-side routing and authentication in a nice way. Or how to deal with authentication and HTTP on the front-end.

So of course, the first task is to find and try out the most popular frameworks for each – Buffalo.io for Golang and Material-UI, React-Query for React work.

Golang as HTTP Server

Like I mentioned above, I gave Buffalo a try. I think I gave it a good run – I did 2 full CRUD-style web apps with it. And it was pretty good! I’d recommend it to anyone who came from the Rails or Laravel community and wanted to get going quickly on Golang. But, I did find it quite heavy for my use-case. There was just a lot of special commands and special layouts to make things work.

There isn’t anything that “didn’t work”. It was more just there were a lot of convention things going on that I didn’t really like.

So in classic Nate fashion, I’ve started building a collection of abstractions on top of gorilla-mux and github.com/markbates/goth. I’ve come up with something that looks sort of like this.

func main() {
	router := r.NewRouter()
	
	// setup database transactions
	router.Use(model.AttachTxHandler("/websocket"))
	
	// setup auth
	router.WithAuth(httpauth.Config{
		// .. setup jwt-based authentication
		// .. oauth setup (optional)
	})

	// serve react-app
	router.ReactApp("/", "./react-app/build", "localhost:3000")

	// simple route
	router.Add("GET", "/api/products", getProducts)
	
	// role-based routing
	router.WithRole(RoleInternal, func(rt *r.RoleRouter) {
		router.Add("POST", "/api/product", todo)	
		router.Add("PUT", "/api/product", todo)	
	})
	
	// api versioning (based on X-APIVersion header)
	router.Versioned("POST", "/api/customer/create",
		r.DefaultVersion(todo),
		r.Version("1", todo),
	)
	
	// rate limiting
	router.Add("GET", "/api/admin/reports", todo, r.RouteConfig{
		RateLimit: &r.RateLimitConfig{
			Count:  10,
			Window: 10 * time.Second,
		},
	})

	// receive a github post-push hook and auto-update ourselves
	router.GithubContinuousDeployment(res.GithubCDInput{
		Path:           "/api/github-auto-deploy",
		Secret:         env.Require("GITHUB_DEPLOY_KEY"),
		PostPullScript: "./rebuild-and-migrate.sh",
	})

	server := httpdefaults.Server("8080", router)
	log.Println("Serving on " + server.Addr)
	log.Fatal(server.ListenAndServe())
}

You can see the full package at github.com/ntbosscher/gobase. It’s far from perfect, but there’s a few designs I’m proud of.

HTTP Handlers

This is where I spend a lot of time, so this part was pretty important to me. There’s a couple features I’ve used.

  1. All handlers return their response. This makes the handler funcs feel a lot more natural IMHO.
  2. In general, failures panic / recover rather than if / err. I get that this is a bit contentious in the Gopher community, but it works great for me. And consider this – why does the http package auto-recover on a per-request basis.
  3. All db commands in a request run in the same transaction. (There’s ways around this for special cases). If an http success code is returned the transaction is committed. (db stuff is a wrapper on github.com/jmoiron/sqlx)
type User struct {
	ID        int
	FirstName string
	Email     string
	CreatedBy int
	Company   int `json:"-"`
}

func CreateUserHandler(rq *res.Request) res.Responder {

	// define the request structure, could have used
	// the User struct here instead.
	input := &struct {
		FirstName string
		Email     string
	}{}

	// parse request body, if fails will return a 400 with error details
	rq.MustParseJSON(input)

	// use the auth-context to get which company/tenant (for multi-tenant systems)
	company := auth.Company(rq.Context())
	currentUser := auth.User(rq.Context())

	// create user using if/err flow
	id := 0
	err := model.QueryRowContext(rq.Context(), `insert into user 
		(first_name, email, company, created_by) 
		values ($1, $2, $3, $4) returning id`,
		input.FirstName, input.Email, company, currentUser).Scan(&id)

	if model.IsDuplicateKeyError(err) {
		return res.AppError("That email is already in the system")
	}

	// fetch user
	// db transaction flows from model.AttachTxHandler through rq.Context() and
	// will be auto committed if we return a non-error response
	customer := &User{}
	model.MustGetContext(rq.Context(), customer, `select * from user where id = $1`, id)

	// returns json with http-status 200 -> { id: 1, firstName: "", email: "", createdBy: 1 }
	return res.Ok(customer)
}

Working with environment variables

This part leverages github.com/joho/godotenv, but basically the usage is this

var connectionString = env.Require("CONNECTION_STRING")
var maxRequestCount = env.RequireInt("MAX_REQUEST_COUNT")
var loggingDest = env.Optional("LOGGING_DEST", "./program.log")

It’s just so must easier than writing all the if/else checks using os.GetEnv

React / Client Side

The client side is less exciting.

Material-UI

I’ve found that Material-UI is an excellent component library. It’s flexible enough for 90% of the projects I work on and makes me 5x more productive. If you don’t use a component library on React yet – use this.

React-Query

React-Query is pretty great for most of my HTTP stuff. Pagination, resource-types stuff is very straightforward. I ended up wrapping it to make a consistent API for my useAsync() helper.

HTTP Data / Actions

I’m sure there’s probably a better way to do this without re-inventing this small wheel. But… I created these two utils to deal with HTTP/Async activity in React components

function UserList() {
   const userList = useAsync((page) => api.users.list(page), [page]);
   return <div>
     {userList.LoadingOrErrorElement || userList.asList.map(u => ...)}
   </div>
}
function CreateUser(props: {onNewUser(user: User): void}) {
   const create = useAsyncAction(async (user) => {
       const newUser = await api.users.create(user);
       props.onNewUser(newUser);
    }, { dependsOn: [props.onNewUser] });
   return <div>
     <form onSubmit={e => {e.preventDefault(); create.callback(user)}}>
        <input ...
     {create.LoadingOrErrorElement}
     <button disabled={create.loading}>Submit</button>
     </form>
   </div>
}

Authentication

I ended up creating a few little things to deal with authentication.

I wanted to block resource requests from spamming the server when we aren’t authenticated. So I created a shared AuthContext that holds “are we authenticated”. To determine our current state, all requests use a shared “fetcher” which detects “access denied” responses.

const isAuthenticated = useIsAuthenticated();
const fetcher = new Fetcher();
await fetcher.get("/api/customers", {limit: 50, offset: 100});

Fetcher is just a wrapper on http-fetch.

Since I use JWT tokens for authentication, we periodically need to refresh the access tokens. To make my life easier, I embedded that functionality in Fetcher. Fetcher will detect when the response is a “access denied” and will attempt to refresh the access token and retry the request.

const fetcher = new Fetcher();
await fetcher.get("/api/customers", {limit: 50, offset: 100});
// -> /api/customers -> 403 access denied
// -> /api/auth/refresh -> 200 OK
// -> /api/customers -> 200 OK -> result