SystemicTS: Runtime Type Validation

Runtime Type Validation is the process of narrowing an object type from unknown into a specific type by examining the object's structure, as opposed to type assertion. It is not a built-in behavior, but for systems, it is essential because the result of its absence, TypeError can start an undefined behavior.

// an example pattern
const maybePerson = await receivePerson();
const decoded = Person.decode(maybePerson);
if (!E.isLeft(decoded)) return;
const person = decoded.right;

Runtime Type Validation is typically needed on an unknown type that the system wants to operate on (e.g. accessing a property, passing it as an argument, etc). It must go hand in hand with The Throwless Pact in the sense that, by default, invalid data detection must not throw.

It is not necessary on type-deleted objects (asserted back into unknown) but is still stored in memory. Unless absolutely necessary, in-memory type-deleted objects are not recommended, instead a polymorphic wrapper (e.g. tagged union) is a better approach.

Implementation

Again, we will see runtime type validation in action using my preferred tool, io-ts and fp-ts, but it should not prevent you from using other tools (e.g. zod).

See the code below.

const user = JSON.parse(await fetchFromUserAPI()) as User;
// May throw `TypeError` because `user` or `user.aliases` could be `undefined`
console.log(user.aliases[0]);

Data that comes from the boundary (e.g. via network, storage) are unknown. However you trust the other party to send the right data, blindly asserting type is fundamentally unsafe. Your program breaks when they screw up. This is the sentiment that began the popularity of runtime validation.

Below is an implementation example of a function that calls an API and validates its returned values.

import * as E from "fp-ts/Either";
import * as t from "io-ts";

// define user, follow the library's docs
// https://github.com/gcanti/io-ts
const User = t.type({
  id: t.string,
  aliases: t.array(t.string),
});
type User = t.TypeOf<typeof User>;

const fetchUser = async () => {
  const maybeUserString = await fetchFromUserAPI(); // returns Either<Error, string>
  if (E.isLeft(maybeUserString)) return maybeUserString; // validate if not error from network
  const userString = maybeUserString.right;

  const maybeJSON = E.tryCatch(
    () => JSON.parse(maybeUserString),
    (e) => e
  );
  if (E.isLeft(maybeJSON)) return maybeJSON;
  const json = maybeJSON.right;

  const maybeUser = User.decode(json);
  if (E.isLeft(maybeUser)) return maybeUser;
  const user = maybeUser.right;

  return user;
};

The above fetchUser is safe and throwless so it can be called this way.

const maybeUser = await fetchUser();
if (E.isRight(maybeUser)) {
  const user = maybeUser.right;
  // `user` is guaranteed to be of type User in this block
}

Tips: Validating Huge Array

Users usually perceive a freeze as "stop running" while it simply runs without updating the user interface.

Unfortunately, innocent validation such as below can freeze the application if the size of the array is big.

const maybeSomeBigArray = t.array(User).decode(json);

The UX trick is to split the validation steps. First, validate the outer array structure. Second, validate the array member and let the consumer control the validation steps.

The outer array structure validation is of the same procedure.

const maybeSomeBigArray = t.array(t.unknown).decode(json);
if (E.isLeft(maybeSomeBigArray)) return maybeSomeBigArray;

The inner validation is a bit tricky because we need to let the caller control its flow. We can leverage a closure to make it something iterator-like.

const fetchAndBatchValidateUsers = () => {
  const maybeSomeBigArray = t.array(t.unknown).decode(json);
  if (E.isLeft(maybeSomeBigArray)) return maybeSomeBigArray;

  return (() => {
    const unknownArray = maybeSomeBigArray.right;
    const initialLength = unknownArray.length;
    const errors = [] as DecodeError[];
    const values = [] as User[];

    // internal methods
    // ======

    const validateNext = () => {
      const item = unknownArray.shift();
      if (!item) return;
      const res = User.decode(item);
      if (E.isRight(res)) {
        values.push(res.right);
      } else {
        errors.push(res.left);
      }
    };

    // methods
    // ======

    // {done: true} when all members are validated, otherwise {done: false}
    const next = () => {
      validateNext();
      return { done: unknownArray.length === 0 };
    };

    // returns a number ranging from 0-1
    const progress = () =>
      initialLength === 0
        ? 1 /* avoid division by 0 error */
        : (errors.length + values.length) / initialLength;

    // Returns done validations as values and errors
    const result = () => ({ values, errors });

    return { next, result, progresss };
  })();
};

The closure exposes next, which validates one item and reports whether it has finished the entire array or not.

Then, the caller can use it this way:

const res = await fetchAndBatchValidateUsers();
if (E.isLeft(res)) return res;
// The closure part starts here
const batched = res.right;
while (!batched.next().done) {
  await updateProgressToUI(batched.progress());
  await sleep(0); // ensure non-blocking behavior
}
await updateProgressToUI(batched.progress());
return batched.result();