Text Field Nullability Defaults

[molomby]

Aka. Why aren't text fields nullable by default in Keystone 6?

Almost all scalar field types in Keystone 6 implement the same behaviour for the validation.isRequired, db.isNullable and defaultValue options:

• validation.isRequired defaults to false
• db.isNullable defaults to the inverse of validation.isRequired (validation.isRequired ? false : true)
• defaultValue defaults to undefined or null

Unless these options are overridden, columns in the DB schema will be nullable and items inserted that don't specify a value for the field – either in the input data or due to a hook being executed – will have a null value. This is the case for the integer, float, decimal, select and timestamp field types. The password type differs only in that it has no defaultValue.

But the text field type options behave differently:

• validation.isRequired still defaults to false
• db.isNullable also defaults to literal false, regardless of the resolved value of validation.isRequired
• If db.isNullable is true, defaultValue defaults to undefined otherwise an empty string

As such, unless db.isNullable is explicitly set to true, the DB columns created will be not nullable with missing values replaced with zero-length strings when items are created.

So why does this inconsistency exist?

To understand why Keystone text fields behave the way they do, we first need to understand some of the issues unique to textual data.

Empty Inputs are Ambiguous

For most types of non-textual data – like numbers, dates, times, etc. – there's an intuitive and obvious distinction between a missing value (ie. null) and any other valid value. If a user is entering, say, prices for products in their ecommerce backend, they're not going to confuse a missing value with a value of 5, 1000 or even 0. A price of 0 indicates the product is free while a missing value (ie. an empty field) likely indicates that data simply hasn't yet been populated.

The same goes for dates and times. There's no special date or time that represents a missing value – even 0000-01-01 00:00:00 or 1970-01-01 00:00:00 (which some may claim are "blank" values) represent real moments in time. If you're filling out a form with a timestamp field, and want to indicate there is no value, you simply leave the relevant field empty.

That, for non-textual data, a field can be left empty (or an existing value removed) to indicate a missing value is intuitive for most users. After all, what else could an empty field mean? A zero-length string submitted for an integer field has no obvious, equivalent value in the set of valid integers. Even select fields are often given an obvious "blank" first option, indicating no value has been selected.

But text fields aren't able to exploit this mechanium. Leaving a text field blank could indicate the value should be null, or it could indicate the correct, known value is a literal zero-length string. After all, zero-length strings are sometimes a perfectly reasonable thing to store.

Entering null Values

So leaving a text field blank is ambiguous; maybe we can make null values explicit?

Although it's technically possible to type a UTF NULL charactor, it's fair to say that users generally won't. There's simply no standard way to enter a null into a text input in a browser.

In Keystone, if you have a nullable text field, the Admin UI will add a checkbox to indicate a null should be submitted, like this:

(screenshot of text field with "Set field as null" checkbox)

If you want this behaviour, simply add db: { isNullable: true } to your text field config. It's super handy for some use cases, specifically, when the text you're dealing with is technical in nature.

Technical text values usually come from or are consumed by software and often have a defined format or schema, for example:

• Complete or partial URLs (eg. URL-friendly slugs)
• Filenames and file paths
• Email addresses
• Ids from remote databases and systems
• External identifiers (Twitter handles, GitHub usernames, etc.)
• Hexadecimal HTML color codes (eg. #1279FC)
• Etc.

For this category of field, storing a zero-length string often isn't helpful. If someone doesn't enter an email address, it's much clearer to indicate the missing value by storing null than by storing an empty string. It usually also makes checking the format of values simpler – validation rules and regex won't be run against null values.

Long term, we're planning to roll out specific field types for these use cases, with "Set to null" mechaniums built in. For example, a dedicated URL field type could add a lot of value by providing a specific UI, default validation rules and other features. An Email field type could correctly account for validation and casing concerns, etc.

However, for a lot of non-technical text data – what I'm going to call natural text data – null values don't really make sense.

Natural Text

Natural text values are parts of the day-to-day language people use to communicate with each other.
They encompasses things like:

• Personal or common names that identify people
• Labels or titles that identify things
• Written descriptions of things
• Dialogue
• Etc.

The concept of null generally isn't used here in these contexts.

For example, say I'm filling out a form of personal information. If I leave the "Middle name" field blank it's obvious to anyone looking at that form that I have no middle name. No one would see that completed form and think, "Huh, this guy has a zero-length string for a middle name!". In these contexts missing and zero-length values effectively have the same meaning.

And this – natural text data – is the primary use case the Keystone 6 text field was designed to address.

Given that, in this context, null usually aren't needed or wanted, the decision was made to replace them with zero-length strings in the default configuration. This gives us a simpler UI, more intuitive filtering and avoids ever accidentally outputting 'undefined' or 'null' when missing values are incorrectly templated or concatenated into a string.

If this isn't the behaviour you want, setting db: { isNullable: true } will make the DB column nullable and give the field a "Set field as null" checkbox in the Admin UI. Note though, it's often best to pair this with validation: { isRequired: true } or a minimum length. Having both zero-length strings and null values in a column can cause unexpected sorting behaviour and complicates filtering – if you want items without a value set you need to remember to check for both conditions, eg:

query {
  users (where: { OR: [{ name: { equals: "" } }, { name: { equals: null } }] }) {
    id
  }
}

If you're also concerned with the nullability of text fields at the GraphQL layer, see also the graphql.isNonNull.read and graphql.isNonNull.[create|update] config options. As always, see the docs for details.

[molomby]

Personally, I'm not convinced the current behaviour is ideal – I'd prefer consistency with developers needing to opt into this special "natural text" behaviour. Maybe we can solve this with additional, more targeted field types in future. Regardless, the above describes the rational for decisions made.

[tofsjonas]

This just feels sooooo much more intuitive. (Also, consistency in code/data tends to cause fewer bugs.)

• validation.isRequired defaults to false
• db.isNullable defaults to the inverse of validation.isRequired (validation.isRequired ? false : true)
• defaultValue defaults to undefined or null

If it is important not to allow empty strings then use defaultValue=''. I would consider empty strings to be more of a frontend issue, not graphql or database.

(I also have to admit that I don't intend to let my end users anywhere near the admin ui, so what happens there is of secondary concern to me)

[tofsjonas]

It feels a bit like I'm always complaining, but I think this package is awesome! (I come from Strapi, but they totally dropped the ball.)

Thank you! 🙏🌷

[tofsjonas]

Btw, is there any way I can set a text() field to be not null without a default value?

[dcousens]

I would have expected defaultValue: undefined to explicitly remove the defaultValue.
I'll see if this can be put in a pull request...

[tofsjonas]

I would have expected defaultValue: undefined to explicitly remove the defaultValue. I'll see if this can be put in a pull request...

Wouldn't that be the same as not setting it at all? I think you'd have to set defaultValue: null, which is not allowed atm 😢

[tofsjonas]

Anywho, what I did was:

• a text() field is NOT NULL by default, but you can override it with db.isNullable

• if a defaultValue isn't set, there is no defaultValue, no empty string or anything

• if you set validation.isRequired it will override db.isNullable and set it to false. An empty string will not satisfy validation.isRequired, you'd have to enter something.

Do I really have to create a whole new field type for this? Doesn't it make sense to have this be the default behaviour? It should satisfy both programmers and end users...

[molomby]

1000 times this.

a: text({ validation: { isRequired: true }, defaultValue: null }) and b: text({ db: { isNullable: false }, defaultValue: null }) should both create columns that are not nullable with no default value (TEXT NOT NULL) as it stands, they both result in TEXT NOT NULL DEFAULT ''.

The only solution right now is to hack up the Prisma schema as @marekryb suggests. These all produce TEXT NOT NULL columns but are pretty gross:

fields: {
  x: text({
    validation: { isRequired: true },
    db: { isNullable: false, extendPrismaSchema: (s) => s.replace(`@default("")`, '') },
  }),
  y: text({
    db: { isNullable: false, extendPrismaSchema: (s) => s.replace(`@default("")`, '') },
  }),
  z: text({
    validation: { isRequired: true },
    db: { extendPrismaSchema: (s) => s.replace(`@default("")`, '') },
  }),
}

Or, if you need this for many columns (as I do) you could use a literal value to indicate field that should have no default then parse them out at the list or schema level. Here I've use the literal string 'null' were I really want null:

fields: {
  x: text({ validation: { isRequired: true }, defaultValue: 'null' }),
  y: text({ db: { isNullable: false }, defaultValue: 'null' }),
  z: text({ validation: { isRequired: true }, defaultValue: 'null' }),
},
db: {
  extendPrismaSchema: (s) => s.replace(/\s+@default\("null"\)/g, ''),
},

Still bad though.

[dcousens]

defaultValue didn't accept null as part of the exported type until #9057, and in that defaultValue: null now results in null as the default value as expected.

Use db.isNullable: true with defaultValue: null

fields: {
  y: text({ db: { isNullable: true }, defaultValue: null }),
},

[tofsjonas]

When it comes to graphql I think the default should be that when something is required, it should be required in createInput, but nowhere else.

If I think of anything I will let you know 😉