Skip to main content

One post tagged with "oss"

View All Tags

· 13 min read
Cristian Petre

A deep dive into Prisma's payload inference, its limitations and potential solutions to some of the uncovered issues.

info

v3.15.1, the latest version of Prisma at the time of writing, was used in the provided code samples.

After extensively using Prisma as my go-to TypeScript ORM for the past few months, I've started noticing how every scenario that required somewhat advanced query patterns was slowly chipping away at the type safety mantra donned by the elegant ORM.

Therefore, if you are a Prisma user, I bring forward the following argument:

Your types are broken.

This is an ideal learning opportunity for someone who interacts with the TypeScript ecosystem on a daily basis. As a result, I have decided to take matters into my own hands and start looking into everything wrong with the generated client types, starting with payload type inference. This led to defining the following criteria for how the GetPayload generics, which are used when you're performing actual queries, should behave.

caution

Some of the criteria are based on the assumption that the query engine's behavior at runtime is correct and intended, while others are based on grounds of consistency and intuitiveness (and might conflict with the runtime behavior).

Premises​

false should not return anything​

Expected mappings:

  • ​ never when it is the exact input type - ideally this would cause the field to be omitted
  • ​ undefined when present in an union of input types

undefined | null should be consistent with the absence of projection filters​

Expected mappings:

  • ​ ModelName when used to directly query a model
  • ​ undefined when used on a relational field
ts
type DirectSelect = ExpectTrue<
  Equal<User, Prisma.UserGetPayload<{ select: undefined }>>
Type 'false' does not satisfy the constraint 'true'.2344Type 'false' does not satisfy the constraint 'true'.
>;
 
type RelationalUndefined = ExpectTrue<
  Equal<
    Prisma.UserGetPayload<{ select: { id: true } }>,
    Prisma.UserGetPayload<{
      select: {
        id: true;
        // oddly enough, this is equivalent to `posts: true` at runtime
        posts: undefined;
      };
    }>
  >
>;
 
type RelationalUndefinedSelect = ExpectTrue<
  Equal<
Type 'false' does not satisfy the constraint 'true'.2344Type 'false' does not satisfy the constraint 'true'.
    Prisma.UserGetPayload<{ select: { id: true } }>,
    Prisma.UserGetPayload<{
      select: { id: true; posts: { select: undefined } };
    }>
  >
>;
ts
type DirectSelect = ExpectTrue<
  Equal<User, Prisma.UserGetPayload<{ select: undefined }>>
Type 'false' does not satisfy the constraint 'true'.2344Type 'false' does not satisfy the constraint 'true'.
>;
 
type RelationalUndefined = ExpectTrue<
  Equal<
    Prisma.UserGetPayload<{ select: { id: true } }>,
    Prisma.UserGetPayload<{
      select: {
        id: true;
        // oddly enough, this is equivalent to `posts: true` at runtime
        posts: undefined;
      };
    }>
  >
>;
 
type RelationalUndefinedSelect = ExpectTrue<
  Equal<
Type 'false' does not satisfy the constraint 'true'.2344Type 'false' does not satisfy the constraint 'true'.
    Prisma.UserGetPayload<{ select: { id: true } }>,
    Prisma.UserGetPayload<{
      select: { id: true; posts: { select: undefined } };
    }>
  >
>;

Payload inference should be distributive​

  • ​ payloads inferred from conditional filters should represent an union of possible outcomes

For example, given this filter where we may or may not select the IDs of an user's posts, the expectation is that posts might be undefined:

ts
type Expected = {
  id: number;
  posts: { id: number }[] | undefined; // not guaranteed to exist at runtime
};
 
type Actual = Prisma.UserGetPayload<{
       
type Actual = {
    id: number;
    posts: Post[];
}
  select: {
    id: true;
    posts: { select: { id: true } } | undefined;
  };
}>;
ts
type Expected = {
  id: number;
  posts: { id: number }[] | undefined; // not guaranteed to exist at runtime
};
 
type Actual = Prisma.UserGetPayload<{
       
type Actual = {
    id: number;
    posts: Post[];
}
  select: {
    id: true;
    posts: { select: { id: true } } | undefined;
  };
}>;

Solutions​

Turn false to nothing​

This one's easy - we just need to return undefined if S narrows down to a false type.

ts
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  U = keyof S
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? "include" extends U
    ? User & {
        [P in TrueKeys<S["include"]>]: P extends "posts"
          ? Array<PostGetPayload<S["include"][P]>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<S["include"][P]>
          : never;
      }
    : "select" extends U
    ? {
        [P in TrueKeys<S["select"]>]: P extends "posts"
          ? TruthyArray<PostGetPayload<S["select"][P]>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<S["select"][P]>
          : P extends keyof User
          ? User[P]
          : never;
      }
    : User
  : S extends false
  ? undefined
  : User;
 
type Payload = Old.UserGetPayload<{
  select: { id: true; posts: true | false };
}>;
type ImprovedPayload = UserGetPayload<{
  select: { id: true; posts: true | false };
}>;
ts
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  U = keyof S
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? "include" extends U
    ? User & {
        [P in TrueKeys<S["include"]>]: P extends "posts"
          ? Array<PostGetPayload<S["include"][P]>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<S["include"][P]>
          : never;
      }
    : "select" extends U
    ? {
        [P in TrueKeys<S["select"]>]: P extends "posts"
          ? TruthyArray<PostGetPayload<S["select"][P]>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<S["select"][P]>
          : P extends keyof User
          ? User[P]
          : never;
      }
    : User
  : S extends false
  ? undefined
  : User;
 
type Payload = Old.UserGetPayload<{
  select: { id: true; posts: true | false };
}>;
type ImprovedPayload = UserGetPayload<{
  select: { id: true; posts: true | false };
}>;

Don't worry about TruthyArray for now, it will make an appearance in the next section.

Consistent undefined | null usage​

A subset of this issue has already been reported - returning the full scalar payload for {select: undefined}.

To better understand why the payload is typed as an empty object, instead of User, let's take a look at the generated type that is responsible for inferring the payload type:

ts
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  U = keyof S
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? "include" extends U
    ? User & {
        [P in TrueKeys<S["include"]>]: P extends "posts"
          ? Array<PostGetPayload<S["include"][P]>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<S["include"][P]>
          : never;
      }
    : "select" extends U
    ? {
        [P in TrueKeys<S["select"]>]: P extends "posts"
          ? Array<PostGetPayload<S["select"][P]>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<S["select"][P]>
          : P extends keyof User
          ? User[P]
          : never;
      }
    : User
  : User;
ts
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  U = keyof S
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? "include" extends U
    ? User & {
        [P in TrueKeys<S["include"]>]: P extends "posts"
          ? Array<PostGetPayload<S["include"][P]>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<S["include"][P]>
          : never;
      }
    : "select" extends U
    ? {
        [P in TrueKeys<S["select"]>]: P extends "posts"
          ? Array<PostGetPayload<S["select"][P]>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<S["select"][P]>
          : P extends keyof User
          ? User[P]
          : never;
      }
    : User
  : User;

The problem here is that as long as the argument S contains a select key, it will try to map over all the truthy keys of that field to determine the payload's structure.

To guard against this unintentional mapping, we could also check whether S["select"] is something we can map over:

ts
    : "select" extends U
    ? S["select"] extends undefined | null
      ? User
      : { ... }
ts
    : "select" extends U
    ? S["select"] extends undefined | null
      ? User
      : { ... }

Because S["select"] is already showing up in a few places, we can clean things up a bit by introducing the infer keyword to create an alias for it.

Furthermore, it doubles as an index signature check, eliminating the need for U.

ts
export type UserGetPayload<S extends boolean | null | undefined | UserArgs> =
  S extends true
    ? User
    : S extends undefined
    ? never
    : S extends UserArgs | UserFindManyArgs
    ? S extends { include: infer IncludeFilter }
      ? User & {
          [P in TrueKeys<IncludeFilter>]: P extends "posts"
            ? Array<PostGetPayload<IncludeFilter[P]>>
            : P extends "_count"
            ? UserCountOutputTypeGetPayload<IncludeFilter[P]>
            : never;
        }
      : S extends { select: infer SelectFilter }
      ? SelectFilter extends undefined | null
        ? User
        : {
            [P in TrueKeys<SelectFilter>]: P extends "posts"
              ? Array<PostGetPayload<SelectFilter[P]>>
              : P extends "_count"
              ? UserCountOutputTypeGetPayload<SelectFilter[P]>
              : P extends keyof User
              ? User[P]
              : never;
          }
      : User
    : User;
 
type ExpectUser = UserGetPayload<{ select: undefined }>;
ts
export type UserGetPayload<S extends boolean | null | undefined | UserArgs> =
  S extends true
    ? User
    : S extends undefined
    ? never
    : S extends UserArgs | UserFindManyArgs
    ? S extends { include: infer IncludeFilter }
      ? User & {
          [P in TrueKeys<IncludeFilter>]: P extends "posts"
            ? Array<PostGetPayload<IncludeFilter[P]>>
            : P extends "_count"
            ? UserCountOutputTypeGetPayload<IncludeFilter[P]>
            : never;
        }
      : S extends { select: infer SelectFilter }
      ? SelectFilter extends undefined | null
        ? User
        : {
            [P in TrueKeys<SelectFilter>]: P extends "posts"
              ? Array<PostGetPayload<SelectFilter[P]>>
              : P extends "_count"
              ? UserCountOutputTypeGetPayload<SelectFilter[P]>
              : P extends keyof User
              ? User[P]
              : never;
          }
      : User
    : User;
 
type ExpectUser = UserGetPayload<{ select: undefined }>;

As we can see, this approach fixes the reported issue. However, it also affects select filters for relational fields, which should be omitted by default:

ts
type ExpectUserId = UserGetPayload<{
type ExpectUserId = {
    id: number;
    posts: Post[];
}
  select: { id: true; posts: { select: undefined } };
}>;
ts
type ExpectUserId = UserGetPayload<{
type ExpectUserId = {
    id: number;
    posts: Post[];
}
  select: { id: true; posts: { select: undefined } };
}>;

To discern between the two scenarios, we're going to need give the type system a helping hand by adding a generic parameter which can be used to perform checks when necessary.

Another consideration is having to unwrap falsy values from the array for to-many relations, for which we can set up a helper type. A similar case could be made for truthy values as well, in part due to dissimilar signatures, but this is beyond the scope of today's investigation.

ts
export type TruthyArray<T> = Exclude<T, undefined | null> extends never
  ? never
  : Array<Exclude<T, undefined | null>> | Extract<T, undefined | null>;
 
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  IsRelationalPayload extends boolean = false
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? S extends { include: infer IncludeFilter }
    ? User & {
        [P in TrueKeys<IncludeFilter>]: P extends "posts"
          ? Array<PostGetPayload<IncludeFilter[P], true>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<IncludeFilter[P]>
          : never;
      }
    : S extends { select: infer SelectFilter }
    ? SelectFilter extends undefined | null
      ? IsRelationalPayload extends true
        ? undefined
        : User
      : {
          [P in TrueKeys<SelectFilter>]: P extends "posts"
            ? TruthyArray<PostGetPayload<SelectFilter[P], true>>
            : P extends "_count"
            ? UserCountOutputTypeGetPayload<SelectFilter[P]>
            : P extends keyof User
            ? User[P]
            : never;
        }
    : User
  : User;
 
type ExpectUserId = UserGetPayload<{
type ExpectUserId = {
    id: number;
    posts: never;
}
  select: { id: true; posts: { select: undefined } };
}>;
ts
export type TruthyArray<T> = Exclude<T, undefined | null> extends never
  ? never
  : Array<Exclude<T, undefined | null>> | Extract<T, undefined | null>;
 
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  IsRelationalPayload extends boolean = false
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? S extends { include: infer IncludeFilter }
    ? User & {
        [P in TrueKeys<IncludeFilter>]: P extends "posts"
          ? Array<PostGetPayload<IncludeFilter[P], true>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<IncludeFilter[P]>
          : never;
      }
    : S extends { select: infer SelectFilter }
    ? SelectFilter extends undefined | null
      ? IsRelationalPayload extends true
        ? undefined
        : User
      : {
          [P in TrueKeys<SelectFilter>]: P extends "posts"
            ? TruthyArray<PostGetPayload<SelectFilter[P], true>>
            : P extends "_count"
            ? UserCountOutputTypeGetPayload<SelectFilter[P]>
            : P extends keyof User
            ? User[P]
            : never;
        }
    : User
  : User;
 
type ExpectUserId = UserGetPayload<{
type ExpectUserId = {
    id: number;
    posts: never;
}
  select: { id: true; posts: { select: undefined } };
}>;

Ideally, fields typed as never would be omitted from the payload. A simple type mapping should suffice, so we won't go any deeper into it:

ts
type Cleanup<T> = {
  [K in keyof T as T[K] extends never ? never : K]: T[K];
};
 
type CleanPayload = Cleanup<{ id: number; posts: never }>;
          
type CleanPayload = {
    id: number;
}
ts
type Cleanup<T> = {
  [K in keyof T as T[K] extends never ? never : K]: T[K];
};
 
type CleanPayload = Cleanup<{ id: number; posts: never }>;
          
type CleanPayload = {
    id: number;
}

At this point, we've got a basic fix in place for the scenarios outlined in undefined | null consistency.

Union types in, union types out​

Distributivity is a property that conditional types exhibit when they perform checks on a generic parameter, and the provided argument is an union type. While this is a very powerful TypeScript feature, it does have limitations.

Nested checks such as the ones we're performing when mapping over the projection filter don't benefit from this behavior by default, as they don't operate directly on the generic parameter.

Some workarounds that come to mind:

  1. extract any logic from the type mapping into a dedicated conditional type and reference it (true distributivity)
  2. explicitly create union types where needed (emulated distributivity)

We continue to build upon the previously proposed solutions, as they also improve payload inference when mapping over conditional filters. In fact, the initial test case for this premise is now passing, and we can focus on other falsy values.

ts
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  IsRelationalPayload extends boolean = false
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? S extends { include: infer IncludeFilter }
    ? User & {
        [P in TrueKeys<IncludeFilter>]: P extends "posts"
          ? Array<PostGetPayload<IncludeFilter[P], true>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<IncludeFilter[P]>
          : never;
      }
    : S extends { select: infer SelectFilter }
    ? SelectFilter extends undefined | null
      ? IsRelationalPayload extends true
        ? undefined
        : User
      : {
          [P in TrueKeys<SelectFilter>]: P extends "posts"
            ? TruthyArray<PostGetPayload<SelectFilter[P], true>>
            : P extends "_count"
            ? UserCountOutputTypeGetPayload<SelectFilter[P]>
            : P extends keyof User
            ? User[P]
            : never;
        }
    : User
  : User;
 
type OriginalTestCase = UserGetPayload<{
  select: {
    id: true;
    posts: { select: { id: true } } | undefined;
  };
}>;
 
type MaybeUndefinedEmail = UserGetPayload<{
             
type MaybeUndefinedEmail = {
    id: number;
    email: string;
}
  select: {
    id: true;
    email: true | false;
  };
}>;
ts
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  IsRelationalPayload extends boolean = false
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? S extends { include: infer IncludeFilter }
    ? User & {
        [P in TrueKeys<IncludeFilter>]: P extends "posts"
          ? Array<PostGetPayload<IncludeFilter[P], true>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<IncludeFilter[P]>
          : never;
      }
    : S extends { select: infer SelectFilter }
    ? SelectFilter extends undefined | null
      ? IsRelationalPayload extends true
        ? undefined
        : User
      : {
          [P in TrueKeys<SelectFilter>]: P extends "posts"
            ? TruthyArray<PostGetPayload<SelectFilter[P], true>>
            : P extends "_count"
            ? UserCountOutputTypeGetPayload<SelectFilter[P]>
            : P extends keyof User
            ? User[P]
            : never;
        }
    : User
  : User;
 
type OriginalTestCase = UserGetPayload<{
  select: {
    id: true;
    posts: { select: { id: true } } | undefined;
  };
}>;
 
type MaybeUndefinedEmail = UserGetPayload<{
             
type MaybeUndefinedEmail = {
    id: number;
    email: string;
}
  select: {
    id: true;
    email: true | false;
  };
}>;

Code branches using false on scalar fields do sound like something we haven't considered until now.

All we're looking for is mapping fields for those union members to undefined, so we'll unite the types ourselves:

ts
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  IsRelationalPayload extends boolean = false
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? S extends { include: infer IncludeFilter }
    ? User & {
        [P in TrueKeys<IncludeFilter>]: P extends "posts"
          ? Array<PostGetPayload<IncludeFilter[P], true>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<IncludeFilter[P]>
          : never;
      }
    : S extends { select: infer SelectFilter }
    ? SelectFilter extends undefined | null
      ? IsRelationalPayload extends true
        ? undefined
        : User
      : {
          [P in TrueKeys<SelectFilter>]: P extends "posts"
            ? TruthyArray<PostGetPayload<SelectFilter[P], true>>
            : P extends "_count"
            ? UserCountOutputTypeGetPayload<SelectFilter[P]>
            : P extends keyof User
            ? User[P] | (false extends SelectFilter[P] ? undefined : never)
            : never;
        }
    : User
  : User;
ts
export type UserGetPayload<
  S extends boolean | null | undefined | UserArgs,
  IsRelationalPayload extends boolean = false
> = S extends true
  ? User
  : S extends undefined
  ? never
  : S extends UserArgs | UserFindManyArgs
  ? S extends { include: infer IncludeFilter }
    ? User & {
        [P in TrueKeys<IncludeFilter>]: P extends "posts"
          ? Array<PostGetPayload<IncludeFilter[P], true>>
          : P extends "_count"
          ? UserCountOutputTypeGetPayload<IncludeFilter[P]>
          : never;
      }
    : S extends { select: infer SelectFilter }
    ? SelectFilter extends undefined | null
      ? IsRelationalPayload extends true
        ? undefined
        : User
      : {
          [P in TrueKeys<SelectFilter>]: P extends "posts"
            ? TruthyArray<PostGetPayload<SelectFilter[P], true>>
            : P extends "_count"
            ? UserCountOutputTypeGetPayload<SelectFilter[P]>
            : P extends keyof User
            ? User[P] | (false extends SelectFilter[P] ? undefined : never)
            : never;
        }
    : User
  : User;

With these changes in place, many of the issues surrounding payload type accuracy should be addressed, allowing more complex access patterns.

ts
type OriginalPayload = Old.UserGetPayload<AdvancedQuery>;
type OriginalPayload = {
    id: number;
    email: string;
    posts: Post[];
}
type ImprovedPayload = Prisma.UserGetPayload<AdvancedQuery>;
type ImprovedPayload = {
    id: number;
    email: string | undefined;
    posts: (Post | {
        id: number;
    })[] | undefined;
}
ts
type OriginalPayload = Old.UserGetPayload<AdvancedQuery>;
type OriginalPayload = {
    id: number;
    email: string;
    posts: Post[];
}
type ImprovedPayload = Prisma.UserGetPayload<AdvancedQuery>;
type ImprovedPayload = {
    id: number;
    email: string | undefined;
    posts: (Post | {
        id: number;
    })[] | undefined;
}

Bear in mind that these solutions are for demonstration purposes, and might have side effects that are not covered by our tests. Furthermore, some payloads might still be incorrect due to the input types not enforing the same degree of strictness as Prisma's underlying query engine. Here's an example:

ts
// the following will result in an error at runtime,
// as the query engine expects at least one truthy key in the select filter.
type ShouldNotBeAllowed = Prisma.UserGetPayload<{ select: { id: false } }>;
             
type ShouldNotBeAllowed = {}
ts
// the following will result in an error at runtime,
// as the query engine expects at least one truthy key in the select filter.
type ShouldNotBeAllowed = Prisma.UserGetPayload<{ select: { id: false } }>;
             
type ShouldNotBeAllowed = {}

This type of payload is not fixable. The solution would be to no longer allow such an input type, but that's beyond the scope of our investigation into payload type inference.

And... that's a wrap. With payloads out of the way, input type constraints are on the menu. A challenge for another day, perhaps.