Buffalo

Cleaning Up Assets in Buffalo

Buffalo doesn't clean up old versions of bundled JavaScript files. This means that the public/asset directory can grow to gigabytes in size, eventually reaching the point where Go will simply refuse to embed that much data.

The tell-tail sign is this error message when you try to run the application:

too much data in section SDWARFSECT (over 2e+09 bytes)

If you see that, deleting public/assets should solve your problem.

Dealing With Public Files

If you want to upgrade your version of Buffalo from 0.15 to 0.18, you may run into a problem with serving public assets.

One of the key differences between these two versions of Buffalo is that public assets — such as JavaScript and CSS — are embedded in the application using Go's builtin file-embedding facility, removing the need to use assetBox. Although it might be possible to continue to use assetBox to embed these files (I haven't tried so I really cannot say), I rather avoid falling behind and make the necessary changes to my app to use Go's native support for embedding files. At the very least, there'll be one less dependency I'll need to worry about.

First, I needed to add a Go file to both the "template" and "public" directory to actually embed the files during the build. Both files are listed below. There are subtle differences between the two, but they essentially do the same thing, which is to make it known to the Go compiler which of the files within these directories should be embedded.

Within the "template" directory, the file looks like this:

package templates

import (
    "embed"
    "io/fs"

    "github.com/gobuffalo/buffalo"
)

//go:embed * */*
var files embed.FS

func FS() fs.FS {
    return buffalo.NewFS(files, "templates")
}

and within the "public" directory, like this:

package public

import (
    "embed"
    "io/fs"

    "github.com/gobuffalo/buffalo"
)

//go:embed *
var files embed.FS

func FS() fs.FS {
    return buffalo.NewFS(files, "public")
}

Second, I needed to configure the renderer to use these embedded files. To do this, I set the value of TemplateFS and AssetsFS within the Render struct in "actions/render.go" to the result of the FS function in the Go files above:

r = render.New(render.Options{
    // [...]

    // fs.FS containing templates
    TemplatesFS: templates.FS(),

    // fs.FS containing assets
    AssetsFS: public.FS(),

    // [...]

After doing this, I discovered that there was one last change to make. It's a sneaky one, as the app will work fine without it in dev, but fall over completely in prod. If you try to deploy it, you'll find the JavaScript and styling are not being applied, and requests for these files return 404.

The last change I made was to replace the app.ServeFile at the bottom of "actions/app.go":

app.ServeFiles("/", assetsBox)

with the call that uses the embedded filesystems:

app.ServeFiles("/", http.FS(public.FS())) 

Once this is done, the embedded files were being served correctly.

There's a bunch of other stuff I needed to do when upgrading Buffalo, like upgrading SCSS and Webpack. I'll leave that for someone else to write about.