Backing up and Restoring Data
BlueMind automatically generates backups so that data can be easily restored through the admin console.
All system data (calendar/contacts/emails) is saved at regular intervals and placed in a system directory. An additional backup system (Bacula, TiNa, etc.) can be used to copy and externalize these backups on tapes or on the external backup system.
Backup data is organized so that you can go back in time and restore user data at a specific point.
Backup frequency can be configured and depends on the disk space you have available. Please note that the minimum time between backups is one day.
Backups can then be linked to a centralized enterprise backup software provider (Time Navigator, NetBackup, etc.).
What are the benefits of a built-in backup system?
- Saves time:
Backups are incremental, only the differences (new messages, deletions, …) transit. - Gains in functionality :
Backup manages data historization. - Space-saving :
Backup saves data without duplicating it. Each backup is standalone (snapshot-like), even if only differences have been transferred. As a result, 10 backups can occupy the same amount of space as your mail storage. - Optimizations :
Data are automatically deduplicated. - Complete backup:
The integrated backup tool ensures that all present and future data is automatically backed up, whether it's e-mail data (stored as files), calendar data, address books (stored as databases) or configuration files. - Backup compatibility :
Backups, regardless of when they were made, remain compatible with all minor versions of the same major release (4.x or 5.x, for example). This means that a backup in a given version can be mounted again in a more recent version of BlueMind. For example, a backup made in 4.8.29 can be remounted in 4.9.25, or a backup made in 5.0.10 can be remounted in 5.2.3. - Performance:
Backups do not cause service interruption; they are designed and carried out within a short timeframe to guarantee data consistency when they are restored. This constraint lies at the heart of the BlueMind backup design. - History :
Data backups must be carried out incrementally, while retaining a history of the data. This historization feature is particularly interesting for those who don't have a large backup infrastructure: data is only present in a single backup, and new data is stored in an increment. In addition, to ensure safe restoration, the program reconstructs the data from one backup and one or several increments. This is transparent for the administrator who is able to specify the date the data should be restored from.
Technical backup architecture
-
Storage and logging component:
BlueMind is based on the Open Source rsync software. This utility allows you to make incremental copies. The data, including the mail spool, is saved by creating symbolic links since the last backup, only new emails are saved. -
BlueMind modules and backups:
BlueMind is based on an architecture that enables services to be split between different servers. Depending on the services hosted on each server, they can be assigned different backup methods adapted to the data stored on that application node (mail spool, database, index, archiving, etc.). As a result, each role assigned to BlueMind servers has its specific backup procedure, thus ensuring optimum and comprehensive data retrieval. -
Backup location :
⚠️ The mount point must at least point to another disk, ideally another server, for the backup to be functional so that backups are not lost with the main data in the event of a disaster. BlueMind offers 2 solutions for configuring storage spaces for backed-up data:
- Setting up an NFS mount on the production server's file system.
- Use a dedicated BlueMind architecture node (server) acting solely as a backup server.
Depending on your availability and type of infrastructure, one or other of these solutions can be used, the aim being in all cases to separate production data from backup data, to ensure that services can be restored in the event of data loss or corruption.
-
Integration into a backup infrastructure:
An information system is often equipped with a backup infrastructure. Software such as Atempo Time Navigator, Tivoli Storage Manager or Net Backup centralize backup methods and manage your backups' historization. These tools interface well with BlueMind. They require a specific configuration to enable the storage of BlueMind's backup history. The BlueMind server performs the backup of production data securely, comprehensively and immediately. The client's backup software must be configured not to historize data. That way, the third-party backup software retrieves the contents of the backup made by BlueMind and transfers it to another tape or other medium.
Backup
Backup configuration
The admin console allows you to configure the number of daily backups you want to keep.
To set up a backup policy, go to BlueMind administration > Backup & Restore > Configuration :
- Retention policy: indicates the number of days during which a backup (here, daily backups) and deleted emails should be kept
- Backup emails (email content): when this option is activated, all server data is backed up, including e-mails. Conversely, when it is not, only contacts, calendar and task data are saved. This allows you to avoid duplicate backups when emails are already backed up in another dedicated system.
- Backup Elasticsearch: when this option is activated, all BlueMind data indexing is backed up. ⚠️ Checking this option will slow down the backup considerably, and the required Minimum disk performance must be respected.
Elasticsearch indexing can be easily recreated from the backed-up data (mails, contacts, appointments, etc.), this option is not essential.
See the limits and best practices section for more details.
Post-backup actions
When the backup is complete, the script /usr/bin/bm-post-full-backup.sh runs automatically if it exists.
It can contain several specific operations to be performed after a successful backup.
In this example, the script /usr/bin/bm-post-full-backup.sh allows creating, for each BlueMind role, a "last" folder pointing to the most recent backup version:
#!/bin/bash
parts=("bm/es" "bm/pgsql" "filehosting/data" "mail/imap" "mail/archive")
server_ip="192.168.124.72"
for part in ${parts[@]}; do
echo "creating last directory for part : $part"
# get last backup directory
last_version=`ls -tr /var/backups/bluemind/dp_spool/rsync/$server_ip/$part | grep -v "last" | tail -1`
echo " last version : $last_version"
# create link
rm /var/backups/bluemind/dp_spool/rsync/$server_ip/$part/last
ln -s /var/backups/bluemind/dp_spool/rsync/$server_ip/$part/$last_version /var/backups/bluemind/dp_spool/rsync/$server_ip/$part/last
done
NFS Mount Configuration
The BlueMind backup space must be shared by all the servers on your BlueMind platform, excluding the edge server.
The backup directory must be accessible from all BlueMind domain nodes. You must therefore perform the following mount and verification operations on all the servers concerned.
For ext3 or ext4 NFS mount points, you need to apply the options nodiratime and noatime to speed up disk access. Advanced file systems such as NTFS, ext3/4 can tell you the last file access date. For each file read, an additional writing operation is performed in order to modify the last file access date and check it. Here is an example of mount for the file /etc/fstab:
# Point de montage NFS
nas.mydomain.lab:/backup /var/backups/bluemind nfs rw,soft,noatime,nodiratime,vers=3,exec 0 0
To enable the NFS mount, run the following command as root user:
mount /var/backups/bluemind
Next, we recommend that you test the mount to make sure it is running properly, using the following command lines - always as root user from the BlueMind server:
cd /var/backups/bluemind
touch test
Then, delete the test file:
rm test
Restore
One single BlueMind backup can be used to perform both a disaster recovery plan and single-user restoration (for a user's data or part of it).
Disaster Recovery Plan (DRP)
For data restoration to work, the server must have the same IP address and the same BlueMind version as the original server.
All the data is restored when you install or re-install BlueMind.
When you install BlueMind using the Setup Wizard, you can choose either to install a blank BlueMind system, or perform a global restoration. A global data restoration allows you to rebuild your entire server from a backup, and is akin to a DRP restoration.
This solution is an easy, fast and safe way for you to rebuild a new BlueMind server.
Single-user Restoration
The single-user restoration functionality BlueMind offers is extremely useful if you want to restore a single user's data quickly. Restoration is done graphically and enables you to choose the object type (entity: user, mail, calendar, shared mailbox, etc.), and then the object whose data you want to restore itself
This functionality also relies on data historization to choose the date of the data you want to back up.
Limits and best practices
The volumes and durations indicated below are indicative values. They can be adjusted according to the performance and capabilities of your platform.
The backup duration is affected by the volume of mail and the retention policy. The larger the mail volume and/or the more important the retention policy, the higher the performance of the backup space.
Generally, for a mail volume <1TB, BlueMind backup can be used without particular constraints.
When the volume of mail is large (>1TB), the BlueMind backup can be used, but we recommend a retention time of 1 day. In this case, the use of third-party tools (file system snapshots, copying to a third-party space, etc.) is recommended to keep a history of backup data.
We also recommend using the double-bottom trash can setting, which is better suited to recovering mistakenly deleted e-mails. In particular, it enables much longer retention times at no cost to platform performance.
For very large mail volumes (several TB), we recommend disabling the backup of emails and ElasticSearch indexing. Backup of e-mails, and ideally of the platform as a whole, will be carried out at infrastructure level (backup of VMs at hypervisor level, backup at file system or storage level, etc.).
In a nutshell:
- Mail restoration mainly relies on the double-bottom recycle garbage can feature
- it is possible to reinject mail recovered from the third-party backup in special cases (mistaken deletion of a mailbox, etc.)
- integrated backup is used for all other data (directories, diaries, address books, etc.), with no retention period constraints, and restoration is carried out via the BlueMind tool. In this context, it is quite possible to store 20 days of data via DataProtect.