What about a chapter on data migrations?

I’m in chapter 23 where you migrate the category names to being unique. It happens that I had two categories im my db that don’t have unique names. For obvious reasons you can’t migrate the database to unique names in that case. You first have to migrate the data and move all acronyms of a duplicate category to the canonical category.

Unfortunately the book doesn’t even seem to mention the topic of data migrations.

It might be a useful addition to show how to deal with data migrations. One would of course also need some unittests for that. To be honest, it sounds like a whole new chapter in the book.
So that’s probably something to be saved for a new edition, or even an “advanced Vapor” book.


After playing around a while, I came up with this. An experienced vapor user could probably improve that tremendously. I don’t really grasp the concept of Futures fully. Nevertheless, it works:

struct MakeCategoriesUnique: Migration {
    typealias Database = PostgreSQLDatabase

    static func prepare(on conn: PostgreSQLConnection) -> EventLoopFuture<Void> {
        return Category.query(on: conn).all().flatMap(to: Void.self) { categories in
            if categories.count == Set(categories.map { $0.name }).count {
                print("No duplicate categories")
                // no duplicate categories
                return conn.future()
            }
            let duplicates = Dictionary(grouping: categories, by: { $0.name }).filter { $1.count > 1 }
            var categoryOperations: [Future<Void>] = []
            for (_, categories) in duplicates {
                let uniqueCategory = categories.first!
                print("Uniquing category \"\(uniqueCategory.name) [ID: \(try uniqueCategory.requireID())\"")
                // get all acronyms for that category
                for category in categories[1...] {
                    try categoryOperations.append(category.acronyms.query(on: conn).all().flatMap(to: Void.self) { acronyms in
                        var acronymUpdates: [Future<Void>] = []
                        for acronym in acronyms {
                            print("Unique category of acronym \"\(acronym.short)\"")
                            acronymUpdates.append(flatMap(to: Void.self,
                                    acronym.categories.detach(category, on: conn),
                                    acronym.categories.isAttached(uniqueCategory, on: conn).map { hasUniqueCategory in
                                        if !hasUniqueCategory {
                                            print("Assign unique category to acronym \"\(acronym.short)\"")
                                            acronymUpdates.append(acronym.categories.attach(uniqueCategory, on: conn).transform(to: ()))
                                        }
                                        else {
                                            print("Acronym \"\(acronym.short)\" already has unique category")
                                        }
                                    },
                                    acronym.save(on: conn)) { _, _, _ in
                                        return conn.future()
                            })
                        }
                        return acronymUpdates.flatten(on: conn)
                    })
                    categoryOperations.append(category.delete(on: conn).transform(to: ()))
                }
            }
            return categoryOperations.flatten(on: conn)
            }.flatMap(to: Void.self) { future in
                Database.update(Category.self, on: conn) { builder in
                    builder.unique(on: \.name)

                }
            }
    }

    static func revert(on conn: PostgreSQLConnection) -> EventLoopFuture<Void> {
        return Database.update(Category.self, on: conn) { builder in
            // uniquing data is destructive, so we can't undo the above data migration
            builder.deleteUnique(from: \.name)
        }
    }
}

@jonasschwartz Can you please help with this when you get a chance? Thank you - much appreciated! :]

@fluchtpunkt that is a good point, thanks for bringing this up. We actually didn’t think the scenario where people had duplicate categories.
However there are actually an example of data migrations, where it adds an admin user, that could be copied/modified for handling categories: vapor-til/User.swift at main · raywenderlich/vapor-til · GitHub

I will try to bring it up as a point for the second edition of the book, but can’t promise anything :slight_smile: