Blackbaud CRM Organizations Risk Copycat Data Breach

Organizations that use Blackbaud CRM are susceptible to a copycat hack for three key reasons:

Copies of unencrypted full database backups of Blackbaud CRM are routinely shared.

Leadership may not recognize the magnitude of the risk from copies of unencrypted database backup files.

The mistaken belief that this is a Blackbaud problem, when in fact it's an internal risk and policy issue.

Blackbaud CRM deployments typically have multiple secondary environments with a full installation of the software and a copy of the database. For example: Staging, Training, Development, and Production Sandbox. These secondary environments get refreshed with a database backup from Production.

If you were to take an inventory of all the unencrypted database backup files being kept informally across all these systems, what would you find?

For each restored database, it's likely your inventory would also uncover at least one copy of an unencrypted database backup file used to restore the database within each of these secondary environments. Once restored, the database backup file doesn't get deleted. Who knows when a backup from a particular point in time may need to be tested?

Backups, it turns out, are terribly handy.

Let's face it, getting database backup copies later on may not be practical or convenient from whomever is in charge of handing out backups. Best to hang onto a database backup copy once you get your hands on one. Don't be surprised if further digging unearths informal repositories kept by various team members, all filled with backups from different times for various needs.

Do you have any developers doing custom programming? Or perhaps team members building out your data warehouse or reporting system? Each team member may have their own database backup copies in any number of places. Now that everyone is working from home, who knows how many places people have squirreled away unencrypted database backup files.

And that's just with your own internal team.

It's not uncommon for backups of Blackbaud CRM databases to be shared with outside vendors and consultants. A copy would likely be found on the FTP server used to transfer the data - either your FTP or their FTP - or both.

Plus the vendor made at least one other copy of your database backup by copying the file from the FTP over to the server of its ultimate destination. Maybe the vendor also maintains an ad hoc file system of customer backups like Blackbaud does? Perhaps that file system uses cloud storage like Dropbox or One Drive? It's not hard to imagine how copies of your database backups could easily multiply into copies of copies on other systems completely outside your control.

Once shared, are those database backup copies ever deleted? Most likely, no. Without an explicit policy delineating expectations for deletion, copies can proliferate and exist for years. At what point does a data breach become an inevitability?

Organizations that use Blackbaud CRM face a sobering truth: they have lost control of their own data.

Since backups are handy and sharing has become routine, trying to stop that behavior is fighting a losing battle. Instead, the database backup files must be encrypted. If a hacker were to gain access to those same database backup files and they are encrypted, the hacker has nothing.

Backup encryption, a feature built into Microsoft SQL Server, would solve the Grand Canyon of risk. SQL backup encryption was added to Microsoft SQL in April 2014 with the release of SQL Server 2014 (12.x).  Multiple encryption algorithms are supported, up to AES 256 bit, which is considered the strongest.

It's free, and very straightforward to setup. Why is encryption still not being used after so many years?

Encryption is inconvenient.

Technology managers tend to want to keep things simple. They care less about risks and more about efficiency. Even if it's free and easy, encryption is an extra step, and therefore perceived to be less efficient.

Presidents and board members oversee insurance and risk, yet rarely understand technology nuances. I'd wager that the vast majority don't even understand what happened in the Blackbaud hack, much less how to combat it.

The tech folks don't want to be bothered. And the leaders figure that if it were important, their fellow leaders at other nonprofits would be doing it already.

Nobody mandates encryption for database backups, much less advocates for it. Ergo, encryption must not be that important.

And here we have the dilemma.

So how do we overcome it?

Concourse subscribes to the “Never Nude” policy when it comes to SQL Server database backups for Blackbaud CRM. A *nude* SQL backup means a database backup file that is not encrypted.

It's not good enough to take an existing unencrypted database backup file then apply encryption to it after the fact. For example, using 7-Zip to encrypt a database backup. Such a method would require starting with an unencrypted *nude* backup to begin with. It also ends with another *nude* backup on the target system once you unzip it.

As you can see, using encryption after the fact does nothing to help reduce the risk of exposure. We still end up with two *nude* backups on two different systems.

Even one nude backup is too nude.

Never nude means just that - no unencrypted database backup files exist at any point in time.

If there are no nude backups, we worry less about data being stolen due to some kind of accidental exposure. If our data backups are always encrypted, the data remains protected even if the files are exposed in a data breach.

If your backup process generates a single uncompressed nude database backup file, you can and should do better. The method outlined below not only adds encryption, it speeds the overall transfer.

First, use compression to reduce the overall size of the backup. Next, break the backup into multiple files, as this allows the backup to be generated much faster than a single file. Plus, once the backup is ready, files may be transferred simultaneously in parallel, thus shortening the overall transfer window. In the end, you'll find that any existing process that requires regular copies of the database benefits from this approach.

And if you're migrating your Blackbaud CRM database to a new hosting environment, you can do so much faster, more efficiently and, of course, securely.

To start with, the SQL Server requires a Database Master Key for the master database. However, the OLTP SQL Server housing the Blackbaud CRM database should already have this key. Nevertheless, I'll show this step for the sake of completeness.

If a Database Master Key already exists (which it probably does), you'll see this error message.

Don't drop the Master Key! As expected, you've already got a Master Key. Hopefully somebody has the key's password saved someplace safe.

Run this against the master database to create the Certificate that we'll use for encryption.

Run this to create an encrypted backup file.

Note that the backup is split into multiple files to speed up the backup process. Other settings such as BufferCount, BlockSize, and MaxTransferSize further optimize the backup speed. Adding compression reduces the size of the backup files, but takes a fair bit of CPU resources.

You may wish to increase the number of files for very large databases. The maximum number of files is 64.

Once the Certificate is in place, it may be used over and over again to encrypt and decrypt database backups.

Should you wish to restore the database to a different SQL Server, export the Certificate and Private Key, and then import them on the target server. The Certificate expires after one year, so you'll only need to perform this step again after the Certificate expires a year later.

To protect the export of the Certificate and Private Key files, generate a random 32-character password. We prefer 1Password where we could also save the password.

Save the Certificate and Private Key to files that are protected by the password generated above.

Next, copy these files to the target server and grant SQL Server access to read the files by running the following command from an Administrator command prompt.

Finally, run this script against the target SQL Server to import the Certificate and Private Key. Of course, include the same password used to export the Certificate.

Restore the database as you normally would using SSMS. If the Certificate was added properly, SSMS will automatically figure that out for you.