made with Findka

Modifying Datomic schema for composite tuples

10 July 2019

Say you define a composite tuple like so:

```clojure
#:db{:ident :foo,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:bar :baz],
     :cardinality :db.cardinality/one}
```
Now suppose you want to change `[:bar :baz]` to `[:bar :quux]`. The [Datomic
docs](https://docs.datomic.com/cloud/schema/schema-change.html) say "You can
never alter :db/tupleAttrs", so simply transacting the following won't work:
```clojure
#:db{:ident :foo,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:bar :quux],
     :cardinality :db.cardinality/one}
```
You'll have to create a whole new composite attribute instead. You might have
the idea to simply do this:
```clojure
#:db{:ident :new-foo,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:bar :quux],
     :cardinality :db.cardinality/one}
```
*Don't do that.* The `:foo` composite tuple is still in Datomic's schema, and it
includes the `:bar` attribute. So if you transact an entity like this:
```clojure
{:bar "testing"
 :quux 123}
```
It will be turned into this:
```clojure
{:bar "testing"
 :quux 123
 :foo ["testing" nil]
 :new-foo ["testing" 123]}
```
This would be especially bad if you're using `:db.unique/identity` for the composite
tuple, like I am in the example. Instead, you should create a new attribute to
use instead of bar:
```clojure
#:db{:ident :new-foo,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:new-bar :quux],
     :cardinality :db.cardinality/one}
```
Conclusion: Any time you need to modify a composite tuple, you should create a
new attribute for the tuple and a completely new set of attributes for
`:db/tupleAttrs`.

Since this is likely to result in a lot of attribute creation/renaming, I've
adopted the convention of appending `-<number>` to the end of attributes that
are just renamings of old attributes, like so:
```clojure
#:db{:ident :foo-0,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:bar-0 :quux],
     :cardinality :db.cardinality/one}
```
Note: I've been doing this in development where:

1. I transact the schema to a dev db, but
2. I haven't yet transacted any other data that actually uses the attributes
(I've only been testing with `d/with`).

If you've already transacted data that uses the attributes, that would
complicate things of course. If you need to make a change to a composite tuple
that's already being used in prod, you'll have to make sure you do things in a
backwards-compatible way. I haven't thought much about this yet.

Alternate solutions, which I haven't tried, include:

 - Set up your dev environment so you don't transact schema or data at all; you
   just use `d/with` for everything (might be complicated and/or annoying).
 - Write some code to easily migrate data from one db to another, then every time you
   want to make a disallowed schema change, move all your data over to a new dev
   db. I think this would work best in combination with the solution I've
   presented above. During development, you rename attributes using the
   `-<number>` convention, but before you deploy to prod, you get rid of all the
   `-<numbers>`s and migrate to a new dev db.

There's more where that came from if you subscribe to my newsletter.