Original now shows 404.

haha i went to go check and they haven't merged a PR since 2017

dig canvas.ucdavis.edu

    [...]
    
    ;; ANSWER SECTION:
    canvas.ucdavis.edu. 1974 IN CNAME ucdavis-vanity.instructure.com.
    ucdavis-vanity.instructure.com. 60 IN A 18.173.121.125
    ucdavis-vanity.instructure.com. 60 IN A 18.173.121.103
    ucdavis-vanity.instructure.com. 60 IN A 18.173.121.15
    ucdavis-vanity.instructure.com. 60 IN A 18.173.121.18

    ;; ANSWER SECTION:
    canvas.duke.edu. 300 IN CNAME duke-vanity.instructure.com.
    duke-vanity.instructure.com. 60 IN A 18.173.121.125
    duke-vanity.instructure.com. 60 IN A 18.173.121.18
    duke-vanity.instructure.com. 60 IN A 18.173.121.103
    duke-vanity.instructure.com. 60 IN A 18.173.121.15

Back when I worked for Instructure ~10 years ago, Canvas was effectively a single, giant, monolithic multitenant app with one instance backed by several thousand app servers and ~100 separate Postgres database clusters that any app server could talk to.

Schools were grouped onto pools of app severs and Postgres database clusters more or less according to locality and cluster availability. I want to say a handful of the largest schools got their own clusters, but I'm not certain, and at any rate their clusters could certainly all talk to each other.

It was actually kind of neat from a technical perspective: any Rails model across the entire Canvas world could have a "foreign key" pointing to any other Rails model anywhere else. Among other things, this allowed for users who could administer multiple Canvas organizations, even if those organizations resided on different Postgres clusters. https://github.com/instructure/switchman is their gem that made that all work. (I put "foreign key" in quotes because the whole thing was implemented in software, not with actual database FKs, for obvious reasons.)

---

Of course, the massive downside to that sort of thing is that if you manage to pop one Canvas app server, you have the keys to the kingdom. I wonder if they'll sharpen the edges between clusters in response to this...

---

(Disclaimer: I left Instructure back in 2017; much could have changed since then, and my memory could be faulty about the specifics. Caveat emptor.)

> Someone dumped the content into a google doc on reddit[1] if anyone's interested.

> [1]: https://docs.google.com/document/d/1MTktVSwTUM5I_w7bKNGj94sT...

Thanks for linking this. Ended up finding my kids school district on the list unfortunately.

https://news.ycombinator.com/item?id=48025001

Funny how a lie is always quicker than the truth...

Don't ransom all your eggs in one basket

My guess would be they get likelihood of getting paid when blackmailing 9,000 schools (at least a few would pay up) than blackmailing Canvas/Instructure.

I don't think any SLA/terms would change who gets to feel the pain.

last I checked it appears grades remain private per planet or so ...

Canvas generally is the 'easiest' to use, and the 'cleanest' looking one although D2L Brightspace is pretty good too. Moodle out of the box is pretty confusing and ugly, but I've seen some heavily customized instances that look a lot better. Blackboard is the worst of the bunch IMO.

In most of these cases, the companies involved did NOT follow standard security practices.

I am pretty sure that is what people mean when they say "held responsible", they mean "held responsible for failing to follow standard security practices", not for the actual act of getting hacked.

I like this analogy, but deaths shouldn't be the leading indicator just an indicator. Family member had a surgery with well known procedures, say removing a gall bladder. Unfortunately, this surgeon skipped a step in lieu of setting a record for fastest procedure. Because steps were skipped, the gall bladder was not scooped into a net to avoid spilled gall stones which resulted stones spilling into the abdominal cavity requiring numerous follow up surgeries to remove the spilled stones as they made themselves known. So clearly not following accepted procedures should be a clear win in a malpractice case, yeah? Wrong. No doctor would testify against the surgeon and the case was dismissed. I feel like this is exactly how it would work in software security incidents as well.

But even if they do everything right, is it really fair to let the companies just shrug their shoulders and say "it happens"? While their users are the ones who really get hurt.

Is that really the analogy you want to use the bolster your argument? Licensing was forced on the medical profession because of rampant quackery causing a large number of deaths. Some of the horrors that went on before enforced medical licensing are well-nigh unbelievable, e.g. https://en.wikipedia.org/wiki/John_R._Brinkley

do you mean equivalent ?.

As a faculty member at a large university…I have a deep respect for the impossible job of university IT departments.

We originally rolled our on LMS decades ago. When we switched to canvas we kept the home brew running for five years past its expiration date because faculty refused to remove their files. Finally each one was manually moved by IT for the recalcitrant old faculty.

The amount of corner cases and performance requirements during rush times (semester start) made it really infeasible for a university to roll their own.

* German universities have this funny system where 51% of such boards are controlled by the professors and the rest is made up of other employees/staff and students. They call it academic participation.

There is a saying in the software security industry that (I'm paraphrasing from rusty memories) a system is secure if the cost of hacking it is higher than the value it protects.

Each system being completely distinct from another means that the cost of hacking the average student goes up by 9000 (from the article, Canvas is used by 9000 schools).

Still not saying that rolling out your own is the preferred solution, but the idea is not as ludicrous as it would seem, and should definitely be entertained and discussed, at least.

Kids from the local uni having a lark, stalkers, vindictive ex employees, local gangs, criminals who understand their victims because they hail from the same community. These are your local hackers. Sift them from the nation states and international crime groups, then deal with the International as a matter of diplomacy. Because we do this so poorly locally, we have little ammunition to when it comes to diplomacy. "reduce attacks by your crime groups and we buy your natural gas, seel you wheat etc"

Want more motivation?- 75% of the local attacks by volume send funds back to terrorist or separatist organizations.

It is not an in-soluble problem. Sentences are a fraction of the answer, effective and receptive reporting processes are more important, then government backing for investigation and enforcement, then policy around home-team activities (ie don't do the bad things yourselves Mr Gov). Deterrence comes after all that.

Really, though, if you want someone to blame, Instructure is not a particularly compelling target. Let's review:

1. Iran is intentionally targeting infrastructure due to a war started by the current administration.

2. China is actively seeking corporate secrets to steal and commercialize for themselves, spurred by extreme protectionism and retaliatory tariffs.

3. North Korea is doing anything they can -- including just taking a remote job by proxy -- in order to extract any money.

4. And Russia is working with and aiding all of them, after everything else going on has forced the embargo to break.

5. All of this while completely alienating every single one of the United States' allies.

6. Meanwhile, the American DHS is currently shut down.

7. And this is after Trump cut funding and personnel for CISA severely enough they've had to end the contract with MS-ISAC, meaning all state and local entities can only remain in the organization if they foot the bill for it directly and CISA and other agencies responsible for cybersecurity are more thinly staffed than they have been in decades.

In short, the current administration systematically disassembled all the protections we have built over the last 100 years, and then placed infrastructure -- schools, in this case, but also power companies, water treatment facilities, communications companies, local governments, hospitals, food producers -- directly on the front lines of the modern geopolitical conflict.

That vast ocean that has kept us safe historically is a poor moat in the modern era.

I do agree with the audit and punishments for clear failure to adhere to established standards.

Just take a look at the recent Epic vs. Health Gorilla lawsuit to see how nonexistent the protection is around exchanging your medical records, one of the most sensitive types of PII.

Instead, there should be standard civil penalties for leaking various degrees of PII paid as restitution to the affected individual. Importantly, this must be applied REGARDLESS of "certification" or whether any security practices were "incorrect" or "insufficient". Even if there's a zero-day exploit and you did everything right, you pay. That's the cost of storing people's secrets.

This would make operating services whose whole "thing" is storing a bunch of information about individuals (like Canvas) much more expensive. Good! It's far to cheap to stockpile a ticking time bomb of private info and then walk away paying no damages just because you complied with some out-of-date list of rules or got the stamp of approval from a certification org that's incentivized to give out stamps of approval.

What? Why? Who died? This whole thing is perfectly dealt with through civil process.

Here’s an example. https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin...

It's very easy to play with lives that aren't yours.

The best response to a cyberattack on critical systems is to take security seriously. Document the offense, avoid the same mistakes and invest in penetration testing. Of course, nobody is incentivized to do that until they're attacked, so the cycle perpetuates itself.

When appropriate. I.e. never.

My highschool, for a while, had a website, which was eventually replaces by a large corporate CMS. Was the website as complicated or complex as the CMS? No, you would have needed to know HTML to publish to it. The CMS was no doubt "more user friendly", I suppose.

But … the original site had a soul. It was unique to the school. There was a student directory! All lost, because the CMS meant utter standardization between all the schools using it (their pages were all identical, except for each got like a different picture of the school as the banner at the top) and the CMS did not do directory anything.

Of course, the directory largely didn't matter in the end. (This was when you needed people's landlines! Quite laughable nowadays…) But it was still sad to see it lost, and several of us students worked on it, which provided us with some early real-world experience.

A large number of my college professors published their own sites, too, where they'd put their lecture notes, homework, etc. I loved those far more than I loved "Canvas" or whatever the ugly LMS we used was.

it's MIT.

There is a lot of people who likely are unaware the latest outage is because they were compromised again.

Them marking the incident as 'Under Maintenance' means the status page isn't reporting this as an outage and adding to downtime%.

As a parent of kids who are impacted by this, I’m not super concerned about the data being held for ransom, but I sure as fuck am concerned about how much it’s going to cost the district to move to another provider.

What seems easy on hobby projects gets way more difficult at scale. Source: experience.

Does a future employer look at pass/fail vs the grade? do they care? Are there even jobs that matter enough to care out there for them?

This seems like, solving the problem but without actually seeing the broader goal or trajectory education is supposed to follow.

Canvas is mostly FOSS

https://github.com/instructure/canvas-lms

Some data was permanently lost, and then officers told reporters that multi-regional backup was not yet built because it was too hard at such a massive scale... of 858 TB.

At my school, tomorrow is the last day of exams. Most of the students have left campus. There's no time or mechanism to schedule an(other) exam.

Backups can be sabotaged (turned off or schedules manipulated) or compromised (say, by lateral movement).

> Even if everything was hosted on Instructure's infrastructure, it's all AWS.

AWS Backup isn't foolproof. Get your hands on administrator credentials as an attacker and suddenly the only thing between everything being gone for good and unrecoverable even for AWS is remembering to have put a permanent deletion protection on all resources in AWS Backup.

Here in the Netherlands a data center's power source (not even the machines) burnt down, data center is offline and University of Utrecht, one of the biggest universities here, is closed. Access passes don't work, work from home environment doesn't work, student information system is down, system for grading doesn't work. No failover for any of them (or maybe it was in the same DC?)

https://nos.nl/artikel/2613485-storingen-in-hele-land-door-b...

Either way, they were under no obligation to adopt this garbage technology regardless of whether it’s available, so this is 110% on them.

Instructure (Canvas's developer) partnered with OpenAI last year [1], about a year after KKR and Dragoneer (PE firms) acquired it [2].

[1] https://www.forbes.com/sites/rayravaglia/2025/07/23/instruct...

[2] https://www.pehub.com/kkr-and-dragoneer-complete-4-8bn-take-...

That's my biggest fear.

Note that little of this really helps the students that it is supposed to help, because as you wisely point out, raw HTML is almost by definition extremely accessible. I work in a field that uses Latex and the source code of Latex should also be considered more accessible than the compiled pdf. But for university administrators the only important thing is that the accessibility metric that appears (or used to appear, before today!) on Canvas shows 100% accessible.