I just want to be able to organize my folders outside of some dumb library system and immich at the time fought that as well.

You can also just use a secure transport layer (like WireGuard or a VPN) instead of relying on every project to implement end-to-end encryption.

That's the objective answer. There's no mystery here. That's exactly how you get what you want and it's not too hard. Not trying to dunk on you or anyone one but this is an easily solved problem, and I think I want to highlight it like this to make sure everyone understands.

Anything web/internet/network service thing, you can add this on. This composability is important to remember in software, this even goes back to "The Unix Way" type stuff.

Edit: regarding cloud based backup, besides the usual privacy and security concerns, you can’t guarantee the fixed price, you might subscribe now and pay for a year, next year you have the typical “oops, operation cost are high we have to raise the prices or shutdown” blog post and now you’re stuck again, download, find another, upload, etc.

Isn't system administration a solved problem now with LLMs? At least for these simple problem domains?

In my setup, I exclusively use the external libraries feature, pointed at a read-only share from my NAS mounted onto my Immich server. (The external libraries are set to resynchronize to the database every few hours). This means I can manage all my media assets myself without worrying about Immich accidentally corrupting them, and if I eventually move off Immich, I just have a single folder of media files organized by date to port around.

The only downside is that I don't directly upload any media files directly to Immich, but that's okay. I have Syncopoli sync files from my phones (on a scheduled cadence) to an intermediary server which organizes and cleans exif data from media files before dropping it into its permanent home on my NAS share. No manual steps to get photos from my phone to my Immich instance!

I had ~200GiB. I selected below 10k files at a time to upload in the web UI (selected all 2014, then 2015). It was fine. More than that many and the UI became unusable.

External Libraries seem like a good option.

They have also recently improved the background import in the Android app so I have heard so that might be worth a try.

Its a bit of a trial, but quite doable. Its likely that things have improved since I did it about 6 months ago.

*Getting the files from takeout* I tried downloading the files onto my laptop via a browser, and then copying them over to my NAS, but quickly gave up. The best approach is to download them directly to the NAS. As you pointed out auth/cookies is an issue. There are multiple ways of solving the issue, but for me I found the best way was to use chromes dev console network view to identify the network request for each file, then right click on it and select "Copy cURL". SSH into my nas and use that command to download the file. There is a bit more info on how to do this here:

https://trog.qgl.org/20241001/downloading-a-google-takeout-f...

*Importing them into immich*

Once I had all my takeout files on the nas, I used immich-go to import them: https://github.com/simulot/immich-go

This took a bit of tweaking to get the right set of command line args that worked well for what I wanted it to do. I also found immich errored out a few times during the import. Fortunately immich-go can just pick up the import where it left of, so I kept re-running it until everything was imported.

*Cleaning it all up* If you just want a huge flat dump of your files your probably good. In my case there were various things I wasn't quite happy about. The default handling of stacking edited images with the originals in albums wasn't what I was after. I wanted to replicate sharing of albums with immich users to match what I had in google photos. For all this kind of cleanup work, I found it quite helpful to work with an AI agent. Give it an API key for your server + the url and get it to help you write cleanup scripts.

See the release note that this HN thread links to ;-)

>>> Background backup improvements Background backup on Android is now significantly more reliable. Previously, the background backup on Android was limited to newly taken photos. Now, the app uses a new periodic task scheduler, which allows you to upload your entire library in the background, and it plays nicer with Android's background execution limits, properly cleans up tasks, and warns you when battery optimization and notification settings might interfere with backups.

On iOS, the background refresh task now runs its sync and upload work in parallel, so uploads actually start within the short time window iOS allows.

"Ente Photos is a paid service, but we offer 10GB of free storage. You can also >>clone this repository and choose to self-host<<."

So both forms...

https://github.com/ente/ente

They really are pretty much just what they appear to be. Im a fan.

If only. It can’t even upload photos any reliably (I self-host). I had it simply fail to upload anything for days (it doesn’t provide any diagnostics, gotta figure out how to build and debug it myself), with no apparent reason. That’s despite keeping app in the foreground, on a charger, for hours, with video uploads and ML features all disabled so it was supposed to focus on just the photos. Server side is fine, web-based uploads work without any issues, app just doesn’t. I haven’t figured it out yet.

> On iOS, the background refresh task now runs its sync and upload work in parallel, so uploads actually start within the short time window iOS allows.

But I don't know if that fixes your issue.

https://github.com/immich-app/immich/discussions/14365

I have all of my photos organized like this: `events -> year/month - holiday -> (album_1, ...)`. and: `home town -> year -> (album_1, ...)`. Photos will be in multiple albums, and there will be edits as well. And I need to track the picked/rejected state as well (and filter on it).

Only reason I haven't moved over to Immich yet is because I am struggling to map my photo organization onto it's way of doing things in a way that's nice. So far my attempts have been unwieldy.

I also imagine that a true E2EE architecture means you have more flexibility with cloud storage, managed hosting, and off-site backups.

I have a multi-region homelab cluster and I share some photos with my friends in the US and my parents in Russia. I’m auto-uploading full library (basically replacing iCloud/Google Photos) and I can share links to selected photos or albums (a reachable node will be determined by a split-view DNS). All without risks of exposing my full photo archive in case either node gets seized or otherwise compromised.

(Now, this is what I’m trying to do. I set things up, but it’s not really functional at the moment, because Ente is buggy af, and I haven’t yet learned how to rebuild and debug their iOS app.)

I do go back and forth on the accessibility tradeoffs of E2EE for average people though. In this scenario, lose or forget your key/password and you lose ALL of your photos which are very important to some people. Losing them is pretty catastrophic. Google Photos or iPhotos really gives people a sense of security about their photos.

ps: It would also make it easier to host cloud instances for Immich without encrypting the file system of a remote server/VPS. Especially when renting servers from small-time sellers, I'm always weary about how much I can really trust their employees access control. I know some level of trust is unavoidable with physical access, but how do they handle those disks during maintenance would also be relevant.

This would force features like semantic search, face detection, video transcoding and thumbnail generation into the clients instead.

Immich assumes trusting the server to have access to your photos is fine. That is always the case when you’re self-hosting.

And I think that’s reasonable, since most users give that trust to Google and Apple.

* Immich doesn't have end-to-end encryption, so I see it for self-hosting (i.e. on hardware I trust, typically at home).

* Ente has end-to-end encryption, which means I can host it on a random VPS.

Two different requirements for two different setups. E2EE adds some complexity, typically to set up a backup somewhere accessible. The fact that I have an unencrypted SDD next to my server at home that my family can grab and access photos is a feature to me: if I disappear I want them to be able to access them.

If you have a server in your homelab where you self-host I mich just encrypt the hard drive with LUKS.

So... yeah, sure, e2ee and encryption and all that but don't wait on perfection when the otherwise situation is pretty dire. It's only encroaching BigTech fueled by surveillance capitalism even more!

To fully encrypt them would just be inviting more problems.

Maybe people have pornography production streams they want to manage using Immmich?

I trust GrapheneOS's security 10x more than my server. Why would I want encryption on messaging, if it's 'just for messaging my grandma'. My data is important to me and I want to keep it secure, even if I don't have a high threat model. E2ee should be the baseline, there's no reason to make security worse on purpose. Encryption in this case is important because it allows defense in depth, it allows others to know their photos are private when using my server and it prevents data access if someone has physical access.

Why trust two devices when one trust one device do trick?

> Or, in the more likely scenario that something happens to your phone, the lack of e2ee means that even if you lost your keys you didn't lose the only memories that remain of your grandma - you just copy across the .jpgs to a new device.

Yes, that's what happens when you lose your keys with e2ee. Every e2ee service is like this. Apple photos, Ente photos, Signal. If I couldn't manage a few words, why would I trust myself to manage a whole server?

The people who care about this disproportionately collect distasteful media and would be in criminal proceedings if their material was uncovered.

In the end who would want all my photos for that amount of work lol?

I "restarted" maybe twice when I first got into Immich to find a good set up for playing reasonably nicely with external libraries, but it does need just a bit more polish. Good to see a sibling comment that it's on their list of priorities!

Hackers

The coolest thing I've done with Immich is just set up Docker on a more powerful machine, run the machine learning libraries, and point my instance (on a low powered server) to use that external computing for face recognition and OCR. Very cool!

And yes, we only look at physical photos or the "on this day X years ago" reels. But to get here, Immich was a valuable tool.

Only it is different 100 photos every year.

This is open source, volunteer developers often focus on doing things that are fun to them, or that scratch their own itch. I can't imagine dealing with Google Photos semi-broken exports is fun to anybody, and you only deal with the pain of the import once, afterwards there's no itch to scratch.

I set up immich backed by ceph last week and I got everything migrated via immich-go, with all metadata albums and all.

Had to change some parallelization option, but otherwise it was a breeze.

Because those services are closed black boxes that don't really let you access them except in a very roundabout way?

Have you thought about backups and redundancy? I was thinking of hosting it on my homelab with backups on some cheap cloud storage but almost any cloud storage ends up costing somewhat similar to Google One so I've been a bit reluctant.

A good backup story is probably the only thing keeping me from switching.

IMMICH_VERSION=v3

docker compose pull

docker compose up

No issues so far. The web UI is not significantly changed, the biggest improvements are on mobile.

It's not necessarily more secure than managed providers either, simply because you are probably not a security engineer, and have far less resources to secure your server. It does prevent Google/iCloud from scraping your data, but it certainly does not mean Hetzner can't access your data. They control the overarching hypervisor and control plane managing your servers/VM's, so there's no way to know what capabilities have been implemented. The majority of what intelligence agencies are capable of has not been leaked or documented publicly.

True E2EE would be that all data on those disks is encrypted by the client your family uses, so even if you combed through the disk volume you would only see cipher text

But as you can see in the linked release note, they made a lot of change to background sync on Android and iOS. It should (TM) work out of the box now.