The History of Ren Bot

Four and half years later, I outline some of the adventures behind Ren bot with respect to the SFU Anime Discord server, and where we're headed next.

The Transition Period

In mid-2017, while we were still ramping up on creating cogs, the upstream project began to work on Red-DiscordBot v3. At the time, we didn’t really pay attention to these upstream changes, as it was still pretty unstable.

The announcement of a stable release of Red v3 on the Red – Discord Bot server.

Fast forward to January 2019, and the upstream released a stable version of Red-DiscordBot v3. With that, the V2 project was on life support, which meant that we would be left on our own if we didn’t upgrade. As with many things, the important question of “Should we upgrade if it ain’t broke?” came up. We had working cogs, so why should we bother?

Upgrading has a lot of benefits. First, that meant we would implicitly have support from upstream, which meant that weird things that were happening to the main core bot could be fixed by them, or we could report them upstream and have way more people look at a problem then say only ourselves. Second, it meant we would have other cogs we could reference to build things. We had stuff for v2 as well, but having them for v3 is also a good thing to have.

On the flip side, upgrading has its pain points. First, v2 and v3 are fundamentally different. v2 uses a discord.py library that is v0.16.x, while v3 uses discord.py v1.x. Switching between the two requires breaking changes, which ultimately boiled down to the fact that it required a rewrite of all the cogs in order to use v3. This meant that lots of time would need to be invested to convert cogs. Second, there was only one Rengineer at the time: myself. sedruk had stepped away from the SAD community, so cog conversions would all have to be done by myself unless I had someone else help me.

Despite everything, we created a v3 branch in our GitHub repository to test out the waters. Just slightly before 2019, there were some SAD members that showed interest in helping out with Ren. One of them was KaguneAstra (or Kagune for short). He came on to test the waters in Fall 2018, and although I still considered myself still pretty new to this, I helped bring him up to speed on our bot set up. It was actually a good time for him to join (from my perspective) because then I would have someone else that would know how v2 stuff worked and how to port things over.

Let the Port Begin!

February 2019 came along, and we began to port things over. For me, the first thing I began to work on was Catgirl, but I didn’t merge it into Ren (but I did leave it on our cog repo). While I left that on the backburner, we also began to port over our favourite cog, WordFilter.

Although a direct port of a cog is pretty straightforward, we opted not to do it “as is” for a few reasons. First, the new version of the bot came with a bunch of new helpers, such as Config. Config takes care of storing persistent data that are required for the operation of your cog. To compare with before, we used to have to manually create everything ourselves and task ourselves with writing everything back to disk. Using Config, we can, for example, tell it to store a value “xyz” in server ABC for user 123, and it’ll take care of it for the disk access for us. Second, our old cogs made assumptions of where data was stored. Because v3 allowed you to start your bot from anywhere on a server (as opposed to browsing to a specific directory), they standardized the location of where data was stored. If we ported things as is, then it could potentially lead to things being stored all over the place. Since third party cogs would be following the same principle, it would make our cogs reusable as well. In the end, making these changes was a step in the right direction.

While working on WordFilter, I had Kagune work on porting the RSS feed. As one of his first projects, he learned about the awful code we previously wrote and faithfully ported it over. With that, he got our first ever v3 cog working and we had it replace our v2 variant.

Cleaning Up Old Code

After getting the first two cogs converted, I realized that we weren’t in a huge rush to move things over. As such, we took the time to also clean up some of the underlying code in v2. As more time was invested into Ren, we learned more and more about Python and its coding standards and idioms. I actually enjoyed looking back at some of my old code and laughing at some of really random things I did, like making really large loops or having C-like boolean comparisons like if myVariable == False. Needless to say, we scrubbed some of that and refactored things to make it more readable.

I’m really glad my past self left some decent commit messages. At times, I would reference back to older commits to see what I had done, and having some of these definitely helped. Crafting good commit messages takes practice, and I think we’re moving in the right direction.

Some decent commit messages on what I was doing at glance.

Aside from refactoring old v2 cog, we did make some quality of life enhancements within v2. Highlights received the ability to change the interval timeout period before notifying via DMs, and replaced a channel mention with the actual channel name, as it previously showed up as a bunch of numbers if you saw it from a mobile notification.

Cog Migrations, continued

After making refactoring changes, we continued moving along with migrations. As August 2019 rolled by, I finished with school, and had more time to work on Ren. We ended up migrating Highlights, SFU, Birthday, Respects, and RoleAssigner over that month.

I took a break in September, as I was in Japan and Hong Kong for a long awaited vacation. When I came back, I had a few weeks before I started my full time job in October. This meant that I had less time to contribute to the bot, except on weekends. 2020 rolled along, and not a lot of progress was made. At that time, we were left with only a handful of higher priority cogs that needed to be migrated, including Welcome and Tags.

Tags: Back for Round Two

As one of the most heavily used cogs, Tags also had to be migrated to v3. The task was slightly daunting, as we had accumulated over 2000 tags and it was tightly coupled with the Alias cog. Because this wasn’t a cog we made from scratch (recall that we obtained it elsewhere), we decided to see if the original author had ported it over. We found that they did, but it required an extra database and additional modifications to make it work with our current implementation. In order to make the transition smoother and avoid additional work, we made a special exception, and more or less ported it over “as is”. Tags v2 was using a version of Config that was similar to v3 in that data saving was abstracted away, so we bundled that version of Config with Tags, and only had to rewrite the parts where we were using the new discord.py API calls.

Although the changes were pretty mechanical, this was one of the cogs where we put out a test version on SAD to try out over the course of two weeks in May 2020. We didn’t receive any reports of issues, so we migrated it over, and with that, one less cog to worry about on v2.

An announcement on testing out tags after the preliminary conversion.

After migrating Tags, we did add new features. We added the tiering capability that allowed users to add different number of tags depending on the role they had. A few months later, we also added the ability to rename tags so that users could keep tag statistics instead of having to delete a tag and recreate it.

We did run into some issues after moving Tags to v3. If you recall, the Config helper from Red v3 helps bundle bot data into a single place. When we ported it over, we forgot about that assumption, so when time came where we had to restart the server and then the bot, all our tags disappeared! We quickly patched a fix for that, and back we were.

YOURLS

Aside from helping out with Ren-related things, we also launched our own instance of Your Own URL Shortener, or YOURLS for short. I use YOURLS for my personal stuff, so it only made sense to do it for the club as well. In April 2020, we had it up and running, where it’s still being used today in announcements or emails.

The Derailment of v2

In July 2020, although the majority of cogs were migrated over, we still kept our v2 instance running, as we had old spoilers, the triggered cog, and ranks still running. On July 8th, we actually noticed the v3 bot crashing when TempChannels was creating the text channel for #dark-hour.

Rengineers debugging the issue and receiving the downtime notification shortly after.

As we looked through the logs, we noticed that Discord updated some of the data they returned in permission overrides, namely two new attributes allow_new and deny_new, which broke the bot because it used a namedtuple and didn’t know how to deal with these new attributes. Checking the discord.py library we used, we discovered it was fixed just two hours prior in a subsequent release. Seeing as Red-DiscordBot needed a bump as well, we filed a PR in to get it merged, and we were back in business!

My first (albeit super small) contribution to the upstream project.

A week or so later, we noticed a large number of errors coming from our v2 instance, namely some API errors, that caused the bot to go into a reconnection loop. We killed the bot and began to root cause it. To nobody’s surprise, it was the same issue as the one we saw before. Unfortunately in this case, we must recall that v2 was pretty much dead at this point. The same can be said about the old discord.py library. As such, we had to manually patch a fix into a local copy of the discord.py library (here’s a YouTube video that is slightly relatable to this scenario) and point our bot to the locally patched version before it would start up properly.

Manually patching the old async branch of the discord.py library, which Ren v2 uses.
Updating Ren v2 to use our patched discord.py library.

A few hours later, I also got emails for two test bots I still had running on v2 about having their tokens reset because I forgot to patch those bots with the fix:

An email from Discord about resetting my bot token.

This encounter made it clear that we needed to ditch the v2 instance soon. Maintaining a library on your own that is falling into pieces (AKA code rot) is just not sustainable.

Adding Rengineers

While v2 remained stagnant, work on v3 continued moving along. Kagune added a few bug fixes for Highlight with spoilers, and we added at-mentions for birthdays. As this was going along, we had other members expressing interest in working on Ren. They included TheDarkBot and Kazuma, and by the the start of 2021, we had two extra people helping us on various bot improvements, which was very welcomed! Dark helped us refactor some of the confusing Birthday logic that we had before, and Kazuma helped us port over the meme Triggered cog.

Saying Goodbye

An announcement post I made about Ren v2 (AKA Red-DiscordBot v2) dying, prior to writing this post.

After porting over the last few lower priority cogs, we finally pulled the plug on Ren v2. With that, we no longer needed to run the two v2/v3 hybrid, and it brought an end to the v2 era of the bot. Thank goodness 😀

Injabie3
Injabie3

Just some guy on the Internet that writes code for fun and for a living, and also collects anime figures.

Articles: 265

3 Comments

Feel free to leave a reply