Tutorial 03 / 20 minutes

Change a running process.

You know the drill: you need to ship a small fix, and that means a deploy, a restart, dropped connections, and a nervous few minutes. In Goer, work is done by processes: small workers that each keep their own data and read their own inbox. You can change one process’s code while it’s running, without stopping anything else. In this tutorial you’ll do it twice: first changing only the code, then changing the shape of the data too. The examples come from the order desk you’ll build in the next-but-one tutorial.

When is it safe to swap?

A process handles one message at a time. So there’s a quiet moment between two messages when the old code has finished and the new message hasn’t started. That gap is the safe point, and it’s the only time Goer swaps code.

How does it find the gap? The swap request waits in the mailbox like any other message. When its turn comes, the swap happens. Nothing that was already waiting gets dropped or jumped.

1. Same data, new code

Write the new version as another function in a //go:build goer file. It uses the same State type. Only the rules change. Here, version two words the confirmation differently.

//go:build goer && goer_source

package orderdesk

func OrderV2Init() OrderState { return OrderState{Status: "idle"} }

func OrderV2(s OrderState, m Msg) OrderState {
	switch x := m.(type) {
	case PlaceOrder:
		s.OrderID = x.OrderID
		s.SKU = x.SKU
		s.Quantity = x.Quantity
		s.AmountCents = x.AmountCents
		s.Status = "waiting for stock"
		s.Inventory <- ReserveStock{OrderID: x.OrderID, SKU: x.SKU, Quantity: x.Quantity, ReplyTo: Self()}
	case StockReserved:
		s.Status = "authorizing payment"
		payment := Ask(Port("payments"), AuthorizePayment{OrderID: s.OrderID, AmountCents: s.AmountCents}).(PaymentResult)
		if payment.Approved {
			s.Status = "fulfilling"
			Port("fulfillment") <- FulfillOrder{OrderID: s.OrderID, SKU: s.SKU, Quantity: s.Quantity}
			Port("notifications") <- Notify{OrderID: s.OrderID, Message: "order confirmed with live pricing"}
		}
	}
	return s
}

From Go, look up the new behaviour and install it using the process’s name. The data stays exactly where it is. Messages from now on go to the new function.

v2, _, err := imgo.Executor(GoerModule, "OrderV2", nil)
if err != nil { return err }

err = sys.HotSwap("orders", goer.Behavior(v2))

2. When the data needs a new shape

Sometimes the new code wants a field the old data doesn’t have. Then you write a migration function: it takes the old state and returns the new one. It runs once, at the safe point, and the mailbox stays put while it does.

//go:build goer && goer_source

package orderdesk

type OrderStateV3 struct {
	Inventory   Pid
	Status      string
	OrderID     string
	SKU         string
	Quantity    int
	AmountCents int
	PriceCents  int
}

func OrderV3Init() OrderStateV3 { return OrderStateV3{Status: "idle", PriceCents: 499} }

func MigrateOrder(old OrderState) OrderStateV3 {
	return OrderStateV3{Inventory: old.Inventory, Status: old.Status, OrderID: old.OrderID, SKU: old.SKU, Quantity: old.Quantity, AmountCents: old.AmountCents, PriceCents: 499}
}

func OrderV3(s OrderStateV3, m Msg) OrderStateV3 {
	switch x := m.(type) {
	case PlaceOrder:
		s.OrderID = x.OrderID
		s.SKU = x.SKU
		s.Quantity = x.Quantity
		s.AmountCents = x.AmountCents
		s.Status = "waiting for stock"
		s.Inventory <- ReserveStock{OrderID: x.OrderID, SKU: x.SKU, Quantity: x.Quantity, ReplyTo: Self()}
	case StockReserved:
		s.Status = "authorizing payment"
		payment := Ask(Port("payments"), AuthorizePayment{OrderID: s.OrderID, AmountCents: s.AmountCents}).(PaymentResult)
		if payment.Approved {
			s.Status = "fulfilling"
			Port("fulfillment") <- FulfillOrder{OrderID: s.OrderID, SKU: s.SKU, Quantity: s.Quantity}
			Port("notifications") <- Notify{OrderID: s.OrderID, Message: "order confirmed"}
		}
	}
	return s
}

v3, v3Init, err := imgo.Executor(GoerModule, "OrderV3", nil)
if err != nil { return err }
migrate, err := imgo.Migration(GoerModule, "MigrateOrder")
if err != nil { return err }

err = sys.Migrate("orders", goer.Behavior(v3), goer.Init(v3Init), goer.Migrate(migrate))

Migrate takes three things: the new behaviour, a new init function (for if the supervisor ever has to start the process from scratch), and the migration for the data that’s live right now.

3. Check it worked

Run the order desk with the dashboard and keep an eye on the Actors tab. Four seconds in, the log says live pricing update applied at a safe point, and the next confirmation says “with live pricing”. At six seconds the migration runs.

Through all of it the process keeps the same name and ID, and its message count keeps going up. No gap.

goer run -dash ./examples/orderdesk

Fair question: how do you know the new code is any good? Goer runs process code two ways, interpreted and compiled to machine code, and every language test has to pass both ways. Each test also checks that memory was handed back correctly, so a leak shows up as a failed test, long before production. And the swap itself is tested against a real mailbox: the data survives, waiting messages keep their order, and a migrated process doesn’t slip back to old code after a crash.