Using customer data for testing
Branching of production databases is a relatively new thing that database providers have started offering.
@niledatabase will also provide this, but it is important to be responsible for when to use these features.
There are two types of branching -
1. Branch only production schema, make schema
changes, test, and merge back to production. This is
super useful, and building this integration with GitHub
and the developer workflow is a big plus.
2. Branch with production data and run tests against it.
This is dangerous and should be used only in 1-2%
of the cases. Most testing just needs seeding data
and running integration tests against it. Imagine
every developer branching production data to test
every change!
There are two reasons why copying over production data is valid -
1. You are trying to reproduce a data-dependent bug.
This might require copying over production data to a
secure environment with limited access and removal
of PII data. Even in this case, you want only to copy
the specific customer’s data and not all the data and
ensure it gets deleted after the testing session.
2. You are not dealing with customer data. A good example of this would be for the database’s internal tools and services use or for personal websites.
“But our customers are not in the EU, which has all the compliance requirements. We should be good with branching production data.”
This is not true either. You are responsible for the customer’s data, even if you are a startup. Their data cannot leak or be accessed outside the production environment. Responsible engineering is all about doing the right thing for the customers.
1. Branch production schema 100% of the time
2. Use seed data 99% of the time for integration tests
3. Copy/branch specific customer data to a secure
environment to reproduce customer issues and
ensure to clean them up