Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External backup of NodeBB #12215

Open
stevefan1999-personal opened this issue Dec 4, 2023 · 5 comments
Open

External backup of NodeBB #12215

stevefan1999-personal opened this issue Dec 4, 2023 · 5 comments

Comments

@stevefan1999-personal
Copy link
Contributor

Description

We should be able to get a simple backup/restore procedure externally, by dumping the database into SQL files and saving the config.json and package.json into a gzip file. This will greatly improve the lifecycle experience for operating a NodeBB instance in a safe and secure way.

Part of #12213, also for #12178

Community forum reference

No response

@julianlam
Copy link
Member

julianlam commented Dec 4, 2023

Hi @stevefan1999-personal, it's been awhile since we approached this issue actually. So long that when we first talked about it, Redis was the only database we supported.

Back then we declined to support this for a simple reason: Redis does not have a command accessible to NodeBB to export the database. You can SAVE and BGSAVE, but the actual database files are inaccessible to a properly configured (aka unprivileged) NodeBB instance.

This limitation is still present in the current version of Redis (unless I am mistaken, and there's an export-style command now).

@julianlam
Copy link
Member

MongoDB and psql also have the same issues, although MongoDB I know has the mongodump command which is executable by the unprivileged user. That can be used to export the database into a .bson file for downloading.

I am not certain what approach is possible for postgres.

@oplik0
Copy link
Contributor

oplik0 commented Dec 4, 2023

For redis, you can scan+dump and then restore. It needs some more manual work but should be fine.

Postgres has pg_dump (and pg_dumpall which exports all databasees) similar to mongodump.

However, I'd argue that using DB-specific backup methods for software that supports multiple databases has a fundamental flaw - you can only restore on similar setups. No cross-db migration using backups. While probably slower (and requiring a bit more work), considering that NodeBB already has essentially its own redis-inspired database abstraction layer, using this abstraction for a custom backup solution might be a better idea.

@julianlam
Copy link
Member

The other argument was that it shouldn't be the responsibility of the forum software to handle backup and restore as there is more than adequate tooling to do this at the system level.

Anything we'd do would merely be an abstraction over existing tooling (with exception of @oplik0's suggestion of having a bespoke export format)

@oplik0
Copy link
Contributor

oplik0 commented Dec 4, 2023

Yup. Just writing a script that calls mongodump and rsync's it (or sends in any other way) somewhere along with config.json and package.json is simple enough that I think most people looking into rolling a custom backup solution (instead of relying on some full disk backups from a cloud provider or something) should probably be able to write one. And it really is a lot less work to make something setup-specific than writing a general solution.

So I agree that a wrapper over few db tools and gzip is kind of out of scope, but a bespoke export might be interesting and it could further help decouple NodeBB from the underlying database further (since it'd allow for backup+restore for migrations between DBs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants