Well, I’ve solved it! I now have a web interface (accessible via VPN, although, in principle, I could expose it to the internet) that allows fast, full-text search of all my old emails. Here is the recipe:
notmuch setup
& notmuch new
). This creates a new folder in your maildir directory containing full-text search info.python3 -m pip install netviel
and then ran it via python3 -m netviel
That’s it! This let’s you search locally. I actually did a few more steps because I wanted to containerize this thing so I could run it on my NAS. I’d be happy to go into detail about that too, if you’re interested. One hiccup was that, for some reason, netviel binds to 127.0.0.1 instead of 0.0.0.0, and there is no way to change that without compiling the project yourself. But, I found a workaround for my Docker container where you can use socat bound to 0.0.0.0 to redirect requests to netviel, so that requests from other computers appear local to netviel.
Anyway, that makes it all sound more complicated than it is. I am super-pleased to have solved this problem at last!
Thanks for this! I’m going to try to get this set up. It sounds perfect.
Yes, I’m coming to similar conclusions myself. To be fair, encryption is a configurable option with Mailpiler. But, yes, it is all digested and stored in a mysql database, which is definitely more opaque than plaintext in the filesystem. I might try the mutt + notmuch solution described by @marty_relaxes@discuss.technics.de below. Sounds like it might be a challenge to set up but would work great forever after. I’ll need to figure out how to convert my mbox files to maildir, but Google suggests there are tools for that. Good luck to you, let us know what you ultimately figure out! I’ve been working on this off-and-on for a few months now without figuring our a good solution!
Edit: I guess, if you want fast full-text search, a database will have to enter the equation somewhere, though.
Alas, no! Things seemed to be going well: I got >90k messages imported from my Google Takeout mbox file before the import was interrupted (not mailpiler’s fault). At this point, I logged into the “auditor” account and was able to see my emails and search them. But, then I resumed the import. By the end of today, the import was finished (~150k messages total). When I logged in with the auditor account, I got some error “No search results” and nothing I could do about it. This is actually what happened last time I tried mailpiler, too, now that I recall. All seemed fine, but, it seems, the database got corrupted or something along the way… So, now it’s useless. I might try it one more time over the next few days. I’ll keep y’all posted.
Does mutt have search capabilities? Is it optimized such that it would be effective with large mailboxes? Thanks!
I am currently working on this. Finally got the Docker working and am importing my 15GB mbox as we speak! I’ll post back here about how it works out.
brick it 4 times
I’d be impressed if the battery lasted long enough for that!
I just spent an hour trying to get this installed in a Proxmox VM. No dice. After install, it just boots to the GRUB rescue prompt. Oh well, seems like a cool idea.
Awesome! You too.
Let me know how it works out for you! I’m happy to be able to share this. I was very pleased with myself but had no one to tell haha. I actually have several copies of this set up with each Gluetun instance connected to different countries. Then, changing country is as easy as changing your tailnet exit node!
I have solved this problem! The trick is to use two Docker containers:
Here is an example docker-compose.yml:
version: "3"
services:
gluetun:
image: qmcgaw/gluetun
container_name: gluetun
# line above must be uncommented to allow external containers to connect.
# See https://github.com/qdm12/gluetun-wiki/blob/main/setup/connect-a-container-to-gluetun.md#external-container-to-gluetun
restart: unless-stopped
cap_add:
- NET_ADMIN
devices:
- /dev/net/tun:/dev/net/tun
volumes:
- ./gluetun:/gluetun
environment:
- VPN_SERVICE_PROVIDER=airvpn
- VPN_TYPE=wireguard
- WIREGUARD_PRIVATE_KEY=xxx
- WIREGUARD_PRESHARED_KEY=xxx
- WIREGUARD_ADDRESSES=xxx
- WIREGUARD_MTU=1320
- SERVER_COUNTRIES=United States
# See https://github.com/qdm12/gluetun-wiki/tree/main/setup#setup
# Timezone for accurate log times
- TZ=America/New_York
# Server list updater
# See https://github.com/qdm12/gluetun-wiki/blob/main/setup/servers.md#update-the-vpn-servers-list
- UPDATER_PERIOD=24h
tailscale:
container_name: tailscale
cap_add:
- NET_ADMIN
- NET_RAW
volumes:
- ./tailscale/var/lib:/var/lib
- ./tailscale/state:/state
- /dev/net/tun:/dev/net/tun
network_mode: "service:gluetun"
restart: unless-stopped
environment:
- TS_HOSTNAME=airvpn-exit-node
- TS_AUTHKEY=xxxxxxxx
- TS_EXTRA_ARGS=--login-server=https://example.com --advertise-exit-node
- TS_NO_LOGS_NO_SUPPORT=true
- TS_STATE_DIR=/state
image: tailscale/tailscale
I have found Nginx Proxy Manager to be a huge time-saver for configuring nginx and certbot.
Fair point, but that probably wouldn’t be a “big difference” for a first time user.
(put an “!” in front)
I don’t see Cryptomator in the comparison. Doesn’t it have a similar feature set?
Consider the Remarkable 2. It is a little Linux computer that allows root access by SSH. It’s moddable, can markup pdfs, and is pleasant to write on. If you get one, just get the bare tablet from the manufacturer. Get a folio and pen from Amazon for way cheaper. Also, you can get $50 off with a referral code from someone who already has one.
They don’t exist. Believe me, since I finally put my old LG V20 out to pasture, I’ve been waiting. The closest thing is the Fairphone, but it doesn’t support all US LTE bands.
It’s too bad Android didn’t lean more into it’s relative freedom. Instead, almost all Android manufacturers have followed Apple’s lead closely. I have to believe there’d be a sizeable market for a flagship Android phone with, say, a removable battery, headphone jack, SD card slot, and an easily unlocked bootloader.
Well, if they hadn’t posted here, I, for one, would never have become aware of the issue, and I’m glad I have!