Building xCBC: a CBC to Discord Tool

Published: June 7th, 2023

For a while now, when linking to a CBC post on Discord, instead of generating a link preview using the Open Graph or Twitter meta tags, the link remains an undescriptive string that may hestiate to click on do to concerns around the trustworthiness and value of it.

This, of course, leads to issues when you are in a community that likes to share news sources, if every other source has the preview, then CBC will be last choice to share.

The Problem

On July 7th, 2021, the last known link preview generated by Discord from the CBC website appeared, after that any attempts to post a link to the website directly met with a roadblock.

Since the issue was self-concealing, I didn’t really notice it until early 2022. In May I decided to do some testing to see if the problem was on Discord’s end or CBC’s end. Deciding to make a copy of an article’s relevant meta tags and using them on an alternative domain.

That worked fine, so the issue wasn’t that the tags were incorrect, Discord would render it just fine, which was really annoying becuase if that was the problem the solution would be really easy.

So unfortunately, it seemed something was just failing somewhere in the engima of a process Discord uses to generate previews, or maybe, it was intentional? Dun dun dun…

With this in mind, I tried some of the other CBC domains, I found that cbc.ca, www.cbc.ca, gem.cbc.ca, andd cbc.radio-canada.ca all failed to have a link preview but ici.radio-canada.ca succeed.

Unable to think of anything more to test, I emailed my findings to CBC, after which we commiserated over the unknown solution and the probablity that the issue was on Discord’s end.

The Lightbulb Moment

So a month later, while tinkering with some projects, an idea popped in my head. What if Discord wasn’t the root of all evil?

Wanting to test a theory, I used Node.js and Ngrok to get Discord’s User-Agent: Mozilla/5.0 (compatible; Discordbot/2.0; +https://discordapp.com), then I used it when visiting cbc.ca and well…

It seems the issue wasn’t on Discord’s side after all! On both cbc.ca and www.cbc.ca, we get this response. This doesn’t explain the issues on gem.cbc.ca or cbc.radio-canada.ca but it turns out those both are more easily explainable: invalid meta tags!

Gem seems to just be missing the required tags entirely while Radio Canada seems to initial load the page with empty meta tags to be filled as the page further loads, which just doesn’t work when services are scraping the basic page for the tags.

Armed with slightly more useful knowledge than before, I sent off further information on June 12th explaining what I found to CBC, in hopes they’d solve it.

As of July 7th, I have yet to hear back.

The Solution

So now, what can be done about the issue? I don’t know if my email to the CBC got thrown in the void or just ignored. A real solution would come from them but who knows when or if that will happen?

But that’s quitter talk, I know the pros and the cons of gramming, I can take a shot at some sort of solution.

So I present: xCBC!

The Plan

I am far from the first person to create a service meant to enhance another site, nor am I the first to think of extending a domain for ease of use of the service. In this case the idea is to have users prepend a character to the regular CBC domain when posting a link to an article on Discord, at which point Discord will ask xCBC for it’s meta tags that it will pull either from cache or from CBC using a whitelisted User Agent.

Then, if a user clicks through the link, xCBC will redirect the user to CBC article for proper viewing. Simple!

The Domain (and Project Name)

The first thing to do in such a project, is of course name it. I needed it to be [something]CBC since the branding of it needs to match the domain and the domain has to be a simple addition to cbc.ca.

I decided to go with xCBC because it satisfied the conditions, was close to the Paste shortcut (on desktop), and most importantly the domain was available for registeration.

The First Attempt

My inital thinking for the application is a simple catch-all Fastify server with a Sqlite3 database as the cache and using Node v18’s built-in Fetch to handle the fetching. The cache may be overkill since Discord also caches it’s link previews, but one of my goals is to not over-query CBC’s site, just in case.

After a bunch of bug-testing, I realized that fiddling with Node’s Fetch wasn’t actually work it and instead switched to using Got, sure this was mainly due to my eyes glazing over when I thought about having to implement my own stream consumer but all those extra packages weren’t going to install themselves!

This version did technically work, testing it manually did show me the end results I was looking for. Unfortunately, the time it was taking to fetch the page from CBC, parse it, cache it, then send it off was too long for Discord to stick around for (around 4 seconds).

So there was nothing to do but to refactor and see what I could do to deal with the slow-down.

The Second Attempt

So after the learning experience of the first attempt, I think the second attempt actually went very well. There’s really only a couple of things I changed.

Starting the Reply Eariler

By switch from using Fastify’s builtin reply.send() to reply.raw, I was able to send the first part of the page before fetching from CBC, making it so Discord wouldn’t cancel out of the request so quickly. It was still taking more time than I would have liked to fetch but I don’t think there is much I can do about that.

Ditching the Cache

Well, not entirely. As I was reading through Got’s documentation I found out that it actually has a built-in caching mechanism, which was great as that meant I could pitch my attempt at one and really streamline the flow of the app. Now instead of having to do the freshness logic myself, I just needed to fetch.

With only these changes, I had it working as intended and my bespoke logic down to about the same length as my parameter checking. It’s honestly a bit funny how little of my own code there is in this. I could in the future Remove some dependencies (I am basically using the built-in HTTP.ServerResponse object, and I am sure I could learn to use Fetch. That’s a problem for future me.

The Wrap-Up

All that was left to do was to build the docker image, set it up behind a reverse-proxy (Traefik, of course), and set up a small landing page for those curious enough.

I don’t know what the future holds for this, it may or may not get used. Maybe nobody will care, but I’ll be happy enough knowing that I put my effort towards providing solutions.