Fetching remote stuff with Hugo 0.90+

It’s a marriage made in heaven: the Cloudinary free tier and Hugo Pipes’ new ability to grab remote items.

2021-12-11

Reading time: 4:16 • 959 words

General note: This site’s appearance, configuration, hosting, and other basic considerations will change over time. As a result, certain content on this page could be at variance with what you’re currently seeing on the site, but the two were consistent when this post originally appeared.

Important update: I have updated this article due to the release of Hugo 0.91.0, which made a breaking change in the feature around which my original text was centered. In short: Hugo 0.90.x used resources.Get to fetch remote items but Hugo 0.91.0+ uses resources.GetRemote instead, so I’ve altered this article accordingly. You may wish to check Hugo Issue #9285 to see why the change occurred.

The release earlier this week of version 0.90.0 of the Hugo static site generator (SSG) suddenly made its Hugo Pipes asset pipeline considerably more powerful by adding, for the first time, the ability to fetch and process remote assets. Previously, Hugo Pipes could work with only local assets — i.e., files actually in a Hugo project’s repository. Now, as the 0.90.0 release notes explained:

The support for remote Resources in resources.Get has been a feature in great demand. This means that you can fetch remote files (images, JSON files, RSS feeds, [etc.]) and use them in Hugo Pipes functions as [if] they were local.

My initial reaction was: well, that’s surely cool, but I doubt I’ll ever need that functionality in my own relatively limited use1 of Hugo. Then, yesterday, a tweet by long-renowned Hugo expert Régis Philibert gave me second thoughts:

With @GoHugoIO latest and its supercharged `resource.Get` there’s no more limit to what your favorite SSG can fetch from the cloud!

Any kind of files really, but a full fledged fetch like API is really what I was waiting for.

And boy am I 🤩.
9:21 PM • December 10, 2021 (UTC)

And, lo and behold, it hit me this morning: this new ability by Hugo to fetch remote stuff meant, now, I could use Base64-encoded low-quality image placeholders (LQIPs) in my image-handling shortcode (most recently described in “Go big or Go home? The sequel”).

So what, you say? Well, this is what: now, the shortcode could halve its image-related HTTP requests to my free Cloudinary account.2 Since HTTP requests are render-blocking, it’s always better to have fewer of them.

You see, up to then, the shortcode got each image’s LQIP by having Cloudinary generate a tiny, 20-pixel-wide version of the original image. While that worked well enough for client-side viewing, it also meant that, for each image, I was sending two HTTP requests to Cloudinary: one for the original image and one for the LQIP.

However, starting with Hugo 0.90.x, Hugo Pipes can go right out to Cloudinary, pull down that 20-pixel-wide image, and instantly translate it into Base64-encoded data that fits right into my HTML. Since this is all happening during the site-building process, visitors not only don’t encounter a slowdown in the image-handling but also don’t make that extra HTTP request for a separate file.

Rather than bore you with the entire shortcode, given that I just got through doing that a few posts ago, I’ll simply show you the parts that changed.3

{{/* These two variables are new */}}
{{- $LQIPdata := printf "%s%s%s" $cloudiBase $LQIPholder $src -}}
{{- $LQIP_get := resources.GetRemote $LQIPdata -}}

<div class="relative bg-center" style="background: url(data:image/jpeg;base64,{{ $LQIP_get.Content | base64Encode }}); background-size: cover; background-repeat: no-repeat;" aspect-ratio="{{ $width }} / {{ $height }}">

Here’s what’s going on:

  • The $LQIPdata variable declaration uses concatenation to supply the Cloudinary-based LQIP’s URL.
  • $LQIP_get is where the Hugo 0.90.x magic comes in, as the newly souped-up Hugo Pipes functionality uses GetRemote to pull that LQIP directly from Cloudinary.
  • In the div element, $LQIP_get.Content | base64Encode converts the result of $LQIP_get into Base64-encoded data and uses it as the div’s background.4

Update, 2021-12-18: Hugo 0.90.1 added some error-handling for those cases where a remote resource might not be available after all (hey, even Cloudinary could “blink” occasionally), which would cause the Hugo build to fail when trying to access it.5 Based on a suggestion I received via Twitter from Hugo maintainer Bjørn Erik Pedersen as well as the Hugo 0.90.1 release documentation, here is an ultra-simplified form of a more resilient example:

{{- $LQIPdata := printf "%s%s%s" $cloudiBase $LQIPholder $src -}}
{{- $LQIP_get := resources.GetRemote $LQIPdata -}}

{{ if $LQIP_get.Err }}
	{{ warnf "%s" . }}
	{{/*
	We output an error message.
	*/}}
{{ else }}
	{{/*
	[Handle things normally.]
	*/}}
{{ end }}

(You could handle this more elaborately, of course. For example, you could specify that, if Hugo can’t fetch a desired remote asset, it instead uses a generic, locally sourced asset as an emergency substitute. However, that goes a bit beyond the scope of this post.)

 
For something that I originally thought wouldn’t ring my chimes, this new power in Hugo Pipes turns out to be pretty frickin’ amazing. Whether I’ll find additional uses for it is unclear, but I already consider it a winner. And I can only imagine the cool stuff that it will enable for the real experts out there in the Hugo-verse.

  1. That is, as compared to that of the regulars in the Hugo Discourse forum. ↩︎

  2. For more on how I started using Cloudinary’s generous free tier to host this site’s images, please see also my post, “Transformed.” ↩︎

  3. This is simpler than what I originally put in this post because I’m now excluding some host-specific stuff that’s likely irrelevant to anybody else. ↩︎

  4. If you compare this to the earlier version of the shortcode, please note that I’m using background here rather than the previous background-image. That’s because, this time, the HTML is calling to data rather than an image file. Also, I put the background-size and background-repeat information in the element, rather than using previously set utility classes, because — well, it just worked better that way, and for reasons I honestly don’t know. (Update, 2021-12-18: This now uses inline styling rather than the more convoluted way I did it originally, and thus would run afoul of a truly strict Content Security Policy.) ↩︎

  5. As the Hugo 0.90.1 release documentation says: “Note that if you do not handle .Err yourself, Hugo will fail the build the first time you start using the failed Resource object.” ↩︎

Latest commit (72b5264a) for page file:
2023-05-02 at 8:02:29 AM CDT.
Page history

NEXT   

PREVIOUS