an adult and I tag everything
avatar from kidd.waterfall.social
Stupid Bits From Waterfall's Code Over The Last Eighteen Monthsthelldev -
I'm going to be honest, I know you're meant to use title case in titles but I don't actually know which words you're not meant to capitalise
I was dealing with a bug report in the Discord earlier and it hit me - all currently known bugs are artefacts of some code I wrote over a year ago and shoved in a file, never to be touched again.
Or, well, no, that's a little bit of an exaggeration. But not much - working on Waterfall has taught me a lot over the last year, so I thought, while I'm working on the API rewrite, it might be amusing to go over some of the more stupid bits of code I've written, the bad decisions, and just things that made me laugh.
All the dumb shit here is fixed in the API rewrite.
The bug report that spawned this post is this one, actually. Someone knows they've reblogged something, but the button never turns green. Annoying, since, other than XKit, we were the first Tumblelog site to implement that feature to my knowledge. Let's look at what could be the problem, shall we?
Notes are stored in their own table in the database, and it's currently over a million entries long. If something is meant to be displayed to a user, it comes from this table. In the case of detecting a reblog, it searches a combination - has [blog ID] reblogged [post source ID]? If yes, it'll be green.
The problem here arises from the simple fact that the notes table is not expected to be 100% consistent. In fact, it's the lowest priority part of the site - while obviously it's not desired, it's considered that if some notes don't get logged, that's fine. Avoid it if possible, but if it happens, no big deal. All the data to reconstruct the table is elsewhere anyway. We could nuked it and have every entry regenerated in less than an hour if we needed to.
And there's the problem. Searching the notes table is inefficient and stupid, especially since it's not consistent. What we SHOULD be doing is searching the Posts table, and asking whether that blog has reblogged a post with the same source post. About 5x faster, and significantly more accurate.
NSFW tags are mandatory. Always have been. Side ramble: I've started seeing a couple blogs putting "I define NSFW as..." in their blog descriptions. No, you're wrong, go and read the rules again.
Anyway, the code, right now, has hardcoded checks for specific tags. When we overhauled the tag system a few months ago, we pre-seeded two tags - "nsfw" and "NSFW". These are tag IDs 1 and 2 respectively (others were added to the database in the order they first appeared in posts - so the first actual tag on the site has an ID of 3). If you're logged out, a minor, or just have adult content turned off, the site runs a separate query to get posts for you that deliberately excludes posts tagged with that. Over time, the query has been amended to also include tag 938 in the query, because someone on mobile did "Nsfw" at some point.
You can see where this is going. Every tag variation, it needs to be added manually to the queries. The same is true of DNR and DNI tags - variations in casing keep appearing and have to be added.
This doesn't go into people who use the tag "nsfw gif" or some such, and not the plain nsfw tag - those are completely omitted from the queries and not caught. There's a really, REALLY obvious solution I missed here, and I'm upset it took me so long to realise it.
When the post is loaded, just. Have it check the first four letters of all the tags and see if they match "nsfw", and if it is, skip it and get another one if appropriate to do so. It's so simple. I'm so dumb. Writing this I realise I haven't gotten it set up in the API rewrite to use the same tactic for DNR and DNI yet. I'm an idiot.
3. Upload Limits
This was just dumb on my part. Most people think of megabytes as being 1000 kilobytes. In actuality, it's 1024 kilobytes. A gigabyte is 1024 megabytes, and a kilobyte is 1024 bytes, etc. When I added the upload limits, I completely forgot about this fact, as well as, apparently, the fact that I have a degree in computer science, and used 1000 as the figure.
This is fixed now, but there is still an issue relating to uploading stuff near the limit that's probably some weird base64 encoding issue. Oh yeah, speaking of.
This particular bug related to when you're editing a post rather than making a new one. If you uploaded a gif in the original and go to edit it, the GIF plays normally. But when you hit save, it turns into a static image at whatever frame it was on when you hit save.
And you know what? There's at least three more I should put here. I really want to. There's another dozen I could put here but aren't as interesting. But I've held my head in my hands three times already reading stuff, so I'll leave it there.
The point is - this has been one hell of a learning experience.
yes that’s right, connect to the open wifi access point you saucy little minx
coffee said: #i thought this was robot thell for a sec #but this is 100% real thell
robot thell??? *amused/concerned combo*
2) go on thell's blog and if you see posts tagged "robot testing" that means a bot wrote them
I feel like I have no where to go when it comes to finding an active blogging site. I feel uncomforable with waterfall rn because I heard that the creator is a racist??? As a black person I feel very uncomforable with that. pillowfort isnt very active rn and mastodon is more like twitter when it comes to layout. nothing is really filling the niche for me.
iirc a while back thell (the creator) was a lot less open minded, but since has been working on it, as well as there are PoC on staff actively working to make waterfall a safe place
(a while back being 5-10 years ago i think? either way am remeber him talk about in the support discord before)
https://babyfairy.tumblr.com/post/185282302372/actually-id-really-like-to-address-my-problems actually the person who he was being racist to didn't accept his apology. Tbh blaming on your racism for being in the uk and calling black people "blacks" is a shitty thing, I wouldn't accept it either.
Apologies for the unplanned downtime this morning! We had to do some maintenance to prepare for the API rollout, and it took significantly longer than expected.
This is the first of two backend patches to prep for the overhaul. The second one will be done at the time of the rollout - retrofitting the existing site to handle the data structure changes would be, in effect, wasted work.
There's still a fair bit to do before then, but in the meantime, you should notice some performance improvements. Thanks for your patience!
Was gonna reply to the shill but they blocked me because I made a tweet about how PF's invite system is artificial scarcity
Waterfall only has five staff members, official or otherwise, only one of whom actually codes or works full time. I'm not really sure what point they were trying to make by emphasising that Pillowfort "only" has 6 (and presumably more since that's only the "official" staff?)
Wow it really is just that easy to summon that one pillowfort 'totally not affiliated just really into the platform' shill
There's two or three that are REALLY prevalent. One of them even tried to advertise on one of my tweets, despite my bio literally saying "I made Waterfall"
Oh lmao I just saw their assertion that we've "paid for extra services"
I can assure you we've not, I'm so averse to external stuff in Waterfall that I've literally reimplemented free public code
I literally wouldn't trust an outside agency to touch Waterfall, EVERYTHING here is done in house. Nobody external has been paid outside of lawyers.
Oh lmao apparently that one really prolific pillowfort shill I've been noticing on every tweet mentioning Waterfall is pillowfort staff, wording in some of their tweets gave it away, I just noticed
Whenever someone makes a tweet talking about Waterfall, there's one account that always hops on going "use Pillowfort, here's a free invite". I checked their tweets because I got suspicious and a) not only is there not a single tweet that's not PF related, they b) Explicitly say "thank you, the mroe people we get the better servers we get"
Which at least strongly implies it's a staff member's smurf account.
Maybe I should start being less picky about our advertising being less intrusive.
Community Update - June 2020staff -
In May, we...
Upgraded the servers in preparation of the app
Made more headway on the new UI
As of writing, we have 30,105 users, 49,755 blogs, and 367,979 posts.
Most work right now is related to getting the new UI done. We're happy to report that we've gotten the first set of data from the site to display on there, signifying the first tangible progress, and frankly, giving a big morale boost. It's still ugly as sin, but real progress is being made now. 99% of code written so far is directly transferrable to the app versions too.
As a micellaneous site stat -the Commissions Market is now consistently handling an average of $100 of transaction volume a week. We're hoping that the reduced fees are helping those artists using it. We've also not had any bug reports for a while, which is excellent - we'll be keeping the Alpha label until we've figured out PayPal integration and written commissions though.
On a more serious note, things are a bit fucked at the minute. You can probably infer our stance if you used a calculator on the number at the beginning, but to be explicit on our stance - Waterfall is a safe space to discuss and organise. While we can't legally ignore any warrants or court orders, and so can't guarantee it's perfectly safe, we invite anyone who can make use of it to use Waterfall to post resources. If we DO recieve a warrant, we will fight it where we're able to do so. Our hope is that not being US-based makes it not worth their time to try and get us to comply.
Provided this line is present, Waterfall has not recieved or complied with any warrant in relation to the protests. Should this line not be present, assume we've been forced to hand information over.
While not in a position to protest directly ourselves, our lead dev is taking a break for a couple of days to work on software tools specifically to assist those protesting, but in the meantime, please feel free to use Waterfall for masterposts, or anything else - as a matter of policy, staff will not remove any posts, users, or blogs providing resources or organising protests.
If you are involved in protesting - please stay safe out there.