Backing up and Restoring Data
BlueMind automatically generates backups so that data can be easily restored through the admin console.
All system data (calendar/contacts/emails) is saved at regular intervals and placed in a system directory. An additional backup system (Bacula, TiNa, etc.) can be used to copy and externalize these backups on tapes or on the external backup system.
Backup data is organized so that you can go back in time and restore user data at a specific point.
Backup frequency can be configured and depends on the disk space you have available. Please note that the minimum time between backups is one day.
Backups can then be linked to a centralized enterprise backup software provider (Time Navigator, NetBackup, etc.).
What are the benefits of a built-in backup system?
- Saves time:
Backups are incremental, only the differences (new messages, deletions, …) transit. - Gains in functionality :
Backup manages data historization. - Space-saving :
Backup saves data without duplicating it. Each backup is standalone (snapshot-like), even if only differences have been transferred. As a result, 10 backups can occupy the same amount of space as your mail storage. - Optimizations :
Data are automatically deduplicated. - Complete backup:
The integrated backup tool ensures that all present and future data is automatically backed up, whether it's e-mail data (stored as files), calendar data, address books (stored as databases) or configuration files. - Backup compatibility :
Backups, regardless of when they were made, remain compatible with all minor versions of the same major version. This means that a backup in a given version can be mounted again in a more recent version of BlueMind. For example, a backup made in 5.2.6 can be reassembled in 5.4.0. - Performance:
Backups do not cause service interruption; they are designed and carried out within a short timeframe to guarantee data consistency when they are restored. This constraint lies at the heart of the BlueMind backup design. - History :
Data backups must be carried out incrementally, while retaining a history of the data. The historization feature is particularly useful for those without a large backup infrastructure: one piece of data is only present in one backup and new data is stored in an increment. In addition, to ensure safe restoration, the program reconstructs the data from one backup and one or several increments. This is transparent for the administrator who is able to specify the date the data should be restored from.
Technical backup architecture
-
Storage and logging component:
BlueMind is based on the Open Source rsync software. This utility allows you to make incremental copies. The data, including the mail spool, is saved by creating symbolic links since the last backup, only new emails are saved. -
BlueMind modules and backups:
BlueMind is based on an architecture that enables services to be split between different servers. Depending on the services hosted on each server, they can be assigned different backup methods adapted to the data stored on that application node (mail spool, database, index, archiving, etc.). As a result, each role assigned to BlueMind servers has its specific backup procedure, thus ensuring optimum and comprehensive data retrieval. -
Backup location :
⚠️ The mount point must at least point to another disk, ideally another server, for the backup to be functional so that backups are not lost with the main data in the event of a disaster.
BlueMind offers 2 solutions for configuring storage spaces for backed-up data:
- Setting up an NFS mount on the production server's file system.
- Use a dedicated BlueMind architecture node (server) acting solely as a backup server.
Depending on your availability and type of infrastructure, one or other of these solutions can be used, the aim being in all cases to separate production data from backup data, to ensure that services can be restored in the event of data loss or corruption.
-
Integration into a backup infrastructure:
An information system is often equipped with a backup infrastructure. Software such as Atempo Time Navigator, Tivoli Storage Manager or Net Backup centralize backup methods and manage your backups' historization. These tools interface well with BlueMind. They require a specific configuration to enable the storage of BlueMind's backup history. The BlueMind server performs the backup of production data securely, comprehensively and immediately. The client's backup software must be configured not to historize data. That way, the third-party backup software retrieves the contents of the backup made by BlueMind and transfers it to another tape or other medium.
Backup
Backup configuration
The administration console is used to implement a backup policy. To do this, go to BlueMind administration > Backup & Restore > Configuration:
Retention policy
-
Daily: indicates the number of backups of each mailbox per day. A daily backup saves a view of each mailbox, consisting solely of references to emails in the main storage area.
-
Number of days to retain deleted emails: indicates the number of days for which emails deleted from the double-bottom trash should be kept in the main storage space.
ℹ️ Default value
By default the retention is 90 days to facilitate the restoration of emails when the object storage is externalized¹. During the retention period, Dataprotect exports can be used to retrieve emails and rematerialize links between mailboxes and object storage. ¹ Outsourcing storage to S3 or Scality Ring object storage is highly recommended for large volumes of mail and/or for retention over a large number of days. See the limits and best practices section for more details.ℹ️ Recommended setting
For best results, the number of days of deleted mail retention should correspond to the difference between the number of days defined for the double-bottom trash (DF) and daily backups (SQ). For example, if DF=5 and SQ=7 => enter 2 in the number of days to retain deleted e-mails and make weekly external backups of /var/backups/bluemind.
The retention policy defines the number of days for which daily backups and deleted emails should be kept in the back-up.
The total retention time for an email is the sum of the number of days defined for the double-bottom trash and the number of days for deleted emails. Here, for example, emails are stored for 7 days in the double-bottomed trash, then 90 days in the main storage area, for a total of 97 days.
Backup
-
Backup emails (email content): when this option is activated, all server data is backed up, including emails, in accordance with the retention policy defined above. When this option is deactivated, only contact, calendar and task data are saved. The incremental email backup performed by daily backups is also deactivated. Note that this has no impact on restoration, since this is carried out from the main storage space (retention) via Dataprotect - which remains active when the option is deactivated - and not from daily backups.
💡 When to enable this option?
Enable this option when object storage is outsourced, to make it easier for the third-party tool to copy backups. -
Postgres backup: when this option is activated, a complete database dump is performed, resulting in a longer backup. On the other hand, when it is not, only user data will be saved (users, resources, shared mailboxes, domain calendars, domain address books).
💡 When should I activate this option?
Activating this option is only necessary to run the Disaster Recovery Plan (DRP) procedure. If another platform backup system is in place and can be used as part of the DRP, this option is not necessary. On vSphere HA-protected BlueMind installations, for example, PostgreSQL backup is generally unnecessary.
Post-backup actions
When the backup is complete, the script /usr/bin/bm-post-full-backup.sh runs automatically if it exists.
It can contain several specific operations to be performed after a successful backup.
In this example, the script /usr/bin/bm-post-full-backup.sh allows creating, for each BlueMind role, a "last" folder pointing to the most recent backup version:
#!/bin/bash
parts=("bm/es" "bm/pgsql" "filehosting/data" "mail/imap" "mail/archive")
server_ip="192.168.124.72"
for part in ${parts[@]}; do
echo "creating last directory for part : $part"
# get last backup directory
last_version=`ls -tr /var/backups/bluemind/dp_spool/rsync/$server_ip/$part | grep -v "last" | tail -1`
echo " last version : $last_version"
# create link
rm /var/backups/bluemind/dp_spool/rsync/$server_ip/$part/last
ln -s /var/backups/bluemind/dp_spool/rsync/$server_ip/$part/$last_version /var/backups/bluemind/dp_spool/rsync/$server_ip/$part/last
done
NFS Mount Configuration
The BlueMind backup space must be shared by all the servers on your BlueMind platform, excluding the edge server.
The backup directory must be accessible from all BlueMind domain nodes. You must therefore perform the following mount and verification operations on all the servers concerned.
For ext3 or ext4 NFS mount points, you need to apply the options nodiratime and noatime to speed up disk access. Advanced file systems such as NTFS, ext3/4 can tell you the last file access date. For each file read, an additional writing operation is performed in order to modify the last file access date and check it. Here is an example of mount for the file /etc/fstab:
# Point de montage NFS
nas.mydomain.lab:/backup /var/backups/bluemind nfs rw,soft,noatime,nodiratime,vers=3,exec 0 0
To enable the NFS mount, run the following command as root user:
mount /var/backups/bluemind
Next, we recommend that you test the mount to make sure it is running properly, using the following command lines - always as root user from the BlueMind server:
cd /var/backups/bluemind
touch test
Then, delete the test file:
rm test
Restore
One single BlueMind backup can be used to perform both a disaster recovery plan and single-user restoration (for a user's data or part of it).
Disaster Recovery Plan (DRP)
For data restoration to work, the server must have the same IP address and the same BlueMind version as the original server.
All the data is restored when you install or re-install BlueMind.
When you install BlueMind using the Setup Wizard, you can choose either to install a blank BlueMind system, or perform a global restoration. A global data restoration allows you to rebuild your entire server from a backup, and is akin to a DRP restoration.
This solution is an easy, fast and safe way for you to rebuild a new BlueMind server.
For further information, see the dedicated page Disaster Recovery Plan (DRP).
Single-user Restoration
The single-user restoration functionality BlueMind offers is extremely useful if you want to restore a single user's data quickly. Restoration is done graphically and enables you to choose the object type (entity: user, mail, calendar, shared mailbox, etc.), and then the object whose data you want to restore itself
This functionality also relies on data historization to choose the date of the data you want to back up.
For further information, see the dedicated page Single-user restoration - DataProtect Navigator.
Limits and best practices
The volumes and durations indicated below are indicative values. They can be adjusted according to the performance and capabilities of your platform.
The backup duration is affected by the volume of mail and the retention policy. The larger the mail volume and/or the more important the retention policy, the higher the performance of the backup space.
Backups stored on the server simplify restoration without the need for third-party tools. However, a high volume of e-mails or a long retention time has a major impact on consumption of the main storage space. A company that receives 10GB of emails a day will consume +/- 1TB in main storage to ensure 90-day retention.
Generally, for a mail volume <1TB, BlueMind backup can be used without particular constraints.
When the volume of mail is large (>1TB), the BlueMind backup can be used, but we recommend a retention time of 1 day. In this case, the use of third-party tools (file system snapshots, copying to a third-party space, etc.) is recommended to keep a history of backup data.
We also recommend using the Mail retention double-bottom trash setting, which is better suited to recovering mistakenly deleted mail. In particular, it enables much longer retention times at no cost to platform performance.
For very large volumes of mail (several Tb), we advise you to disable backup of mail and user data (directories, calendar, address book, resources, shared mailboxes, domain calendars, domain address books) during Backup configuration. Backup of e-mails, and ideally of the platform as a whole, will be carried out at infrastructure level (backup of VMs at hypervisor level, backup at file system or storage level, etc.).
Summary:
- mail restoration relies mainly on the Mail retention of the double-bottom trash can
- it is possible to reinject mail recovered from the third-party backup in special cases (mistaken deletion of a mailbox, etc.)
- integrated backup is used for all other data (directories, diaries, address books, etc.), with no retention period constraints, and restoration is carried out via the BlueMind dedicated tool. It is therefore possible to store 20 days of data via DataProtect.