Post submission application schema #306

apricot13 · 2025-02-05T19:55:19Z

May have gone a little overboard with preparing this for different application types!

I have purposely not modified anything in PrototypeApplication and have chosen to reference things in their folders rather than move them to shared/ - this was mostly to make the PR process a little easier and to avoid tricky rebases as this schema is still very much in draft form!

Theres still a lot of work to do around the files but this gives us everything we need in the DPR including comments with room to grow!

Post Submission Application Schema

The Post Submission Application schema is designed to represent an application, post-submission, within an application.

Future iterations will create redacted versions of this, and the PrototypeApplication schema, which will act as a 'published' schema for applications such as the DPR.

The Application Process

flowchart TD

%% stages

submission[SUBMISSION]:::stage
validation[VALIDATION]:::stage
consultation[CONSULTATION]:::stageOptional
assessment[ASSESSMENT]:::stage
appeal[APPEAL]:::stageOptional
highCourtAppeal[HIGH COURT APPEAL]:::stageOptional


%% status

returned:::status
withdrawn:::status
determined:::status
undetermined:::status



%% flow

submission --> validation
validation --> consultation
consultation --> assessment
assessment -.-> appeal
appeal -.-> highCourtAppeal


%% status flow

validation --> |INVALID|returned

assessment --> determined
appeal --> determined
highCourtAppeal --> determined

consultation --> withdrawn
assessment --> |before determination made|withdrawn



classDef stage fill:#655356,color:#FFF,stroke-width:0px,font-weight:bold;
classDef stageOptional fill:#F2D1C9,color:#655356,stroke-width:0px,font-weight:bold;
classDef status fill:#52489C,color:#FFF,stroke-width:0px,font-weight:bold;
classDef check fill:#BAD1CD,color:#655356,stroke-width:0px;

There are a lot of different ways to define the application process, there is the process for the applicant, the agent and the backend processes in BOPs. The DPR is also displaying the information in a specific way as well. I wanted to try and come up with a process that was flexible, future proof and easy to read and discern information from.

A strict rigid structure wasn't going to work too well since there are so many "if this then thats" so this is what I have come up with.

Stages

All applications go through these stages and they pick up various contextual pieces of information along their way

flowchart TD

  %% stages

  submission[SUBMISSION]:::stage
  validation[VALIDATION]:::stage
  consultation[CONSULTATION]:::stageOptional
  assessment[ASSESSMENT]:::stage
  appeal[APPEAL]:::stageOptional
  highCourtAppeal[HIGH COURT APPEAL]:::stageOptional

  %% flow

  submission --> validation
  validation --> consultation
  consultation --> assessment
  assessment -.-> appeal
  appeal -.-> highCourtAppeal



  classDef stage fill:#655356,color:#FFF,stroke-width:0px,font-weight:bold;
  classDef stageOptional fill:#F2D1C9,color:#655356,stroke-width:0px,font-weight:bold;

Submission, Validation and Assessment are all required stages. Whereas consultation, appeal and high court appeal are optional, dependant on other conditions.

The stage an application is in does not determine its status.

Status

flowchart TD

%% status

returned:::status
withdrawn:::status
determined:::status
undetermined:::status

classDef status fill:#52489C,color:#FFF,stroke-width:0px,font-weight:bold;

Each status reflects a higher level state which an application can be in. (This field could also be called state I guess)

An application is returned if it is marked as invalid.
An application can be withdrawn after submission and until a decision is made ie it is determined
An application is determined if any decision has been made about it, this could be assessment, appeal or high court appeal decisions
An application is set to undetermined at all other times

I wanted to avoid being too prescriptive in the status's. There are so many variations that this felt more straight forward. From each of these status' you should be able to quickly determine the rest of the information you need about an application, eg was it an appeal or a council decision that was made.

How we all fit in

flowchart TD

    subgraph PlanX
    submission
    end
    subgraph BOPS
    validation
        subgraph DPR
           consultation
           assessment
           appeal
           highCourtAppeal
        end
    end



%% stages

submission[SUBMISSION]:::stage
validation[VALIDATION]:::stage
consultation[CONSULTATION]:::stageOptional
assessment[ASSESSMENT]:::stage
appeal[APPEAL]:::stageOptional
highCourtAppeal[HIGH COURT APPEAL]:::stageOptional


%% status

returned:::status
withdrawn:::status
determined:::status
undetermined:::status



%% flow

submission --> validation
validation --> consultation
consultation --> assessment
assessment -.-> appeal
appeal -.-> highCourtAppeal


%% status flow

validation --> |INVALID|returned

assessment --> determined
appeal --> determined
highCourtAppeal --> determined

consultation --> withdrawn
assessment --> |before determination made|withdrawn



classDef stage fill:#655356,color:#FFF,stroke-width:0px,font-weight:bold;
classDef stageOptional fill:#F2D1C9,color:#655356,stroke-width:0px,font-weight:bold;
classDef status fill:#52489C,color:#FFF,stroke-width:0px,font-weight:bold;
classDef check fill:#BAD1CD,color:#655356,stroke-width:0px;

The structure

interface ApplicationSpecification<
  TPrimary extends PrimaryApplicationType,
  TGranular extends ApplicationType,
> {
  applicationType: TGranular;
  data: {
    application: Application<TPrimary>;
    localPlanningAuthority: LocalPlanningAuthority<TPrimary>;
    submission: Submission<TPrimary>;
    validation: Validation<TPrimary>;
    consultation?: Consultation<TPrimary>;
    assessment?: Assessment<TPrimary>;
    appeal?: Appeal<TPrimary>;
    caseOfficer: CaseOfficer<TPrimary>;
  };
  comments?: {
    public?: PublicComments;
    specialist?: SpecialistComments;
  };
  submission: PrototypeApplication;
  metadata: PostSubmissionMetadata;
}

data.Application

This is where I have added the reference number for an application.

It also shows the stage and status.

data.localPlanningAuthority

This section includes things about the application that are unique to that LPA. See comments below for more information on this.

data.submission

This contains submittedAt - this is the date which the application was submitted, either through PlanX or for example the date written on the envelope if it was posted.

data.validation

receivedAt is the date that the application is received by the back office system. It may not be the date it was sent, if there have been delays in the submission process.

validatedAt is the date isValid was set, its presence doesn't mean that the application was valid

This includes a flag for isValid, which can be used alongside status to find out when an application was set to returned.

data.consultation?

This includes the beginnings of the sitenotice work for the DSN (currently a boolean).

The startDate and endDate of the consultation period.

Currently there is nothing in here about expiryDate or extensions to the consultation period.

During the consultation period an application can recieve comments.

Not all application types have consultation periods - this is something that will need doing.

data.assessment?

Assessment is when the council and optionally a committee make a decision. councilDecision, committeeSentDate and committeeDecision

If an application is Prior Approval it also has priorApprovalRequired since they can be 'Prior approval required and approved', 'Prior approval not required', 'Prior approval required and refused'

Until there is a decision in this section an application is undetermined

data.appeal?

Appeal's are lodged, started, validated then decided.

While an appeal decision is being made an application is still set to determined.

data.caseOfficer

This is the officer assigned to the application throughout the process.

Comments

We've also proposed the beginning of a structure for comments in this section, See comments below for more information on this.

metadata.publishedAt

I haven't included 'published' as a stage because an application can be published at any time, its also not a status its something else. I have added publishedAt into the metadata to match other metadata submittedAt.

Submission

Rather than repeat the same data for the Post Submission version, I've attached the original submission to this schema, which will have a redacted version at a later date. This makes it so that only additional data, post submission, gets added to this schema. For example Case Officer.

Comments

Comments can be submitted during the consultation period, which can bet determiend by looking in the data.consultation.startDate and data.consultation.endDate fields.

Not all applications have a consultation period, however some LPA's accept comments until a decision is made. I have added the data.localPlanningAuthority.commentsAcceptedUntilDecision flag so that each LPA can set this and this flag will allow the DPR to determine whether or not comments can be sent - without complicating the application process

Appendix

One more diagram!

jessicamcinchak

A huge effort, thanks so much for a thoughful & well-documented contribution! 🌟

Domain knowledge & data of this piece is definitely beyond PlanX and between BOPS & DPR to decide, but left a few minor comments throughout - mostly around "Metadata" and what we actually want to mean by that ! (eg just our specific systems or the application/post-submission process)

Let's keep discussing & please shout if any developer experience/pain points around working in this repo that we can help smooth out 🙂

jessicamcinchak · 2025-02-11T13:30:49Z

examples/postSubmissionApplication/data/lawfulDevelopmentCertificate/proposed.ts

+      publishedAt: '2025-01-10T13:00:00.288Z',
+      organisation: 'BKM',
+      id: '8da51c5b-a2a0-4386-a15d-29d66f9c121c',
+      source: 'PlanX',


Is the source actually still 'PlanX' here (and in other examples)?

at some points yes but not always! I've updated the schema to extend from the base metadata now too so theres more options!

jessicamcinchak · 2025-02-11T13:32:03Z

examples/postSubmissionApplication/data/planningPermission/major.ts

+ * decision and no appeal = stage = assessment, status = determined
+ * determined can refer to the council's decision or any of the appeal decisions
+ *
+ */


This is a helpful pattern, there's always so many possible permutatons when it comes to examples and this is nice and explicit about what's covered ➕

I've updated this a bit more now and have included examples of all the stages for three kinds of application that have some key differences. LDC, PP and PA. It mooches off the PrototypeApplication submissions. 😅

jessicamcinchak · 2025-02-11T13:35:35Z

examples/postSubmissionApplication/data/planningPermission/major.ts

+          sentiment: 'Accepted',
+          constraint: {
+            name: 'Article 4 Direction Area',
+            url: 'https://www.planning.data.gov.uk/dataset/article-4-direction',


nit: for constraints, is it possible to use Planning Data's "entity" URL rather than the general "dataset" here?

For PlanX, this is the difference of being able to say "At least one Article 4 Direction applies" versus "This specific A4 direction applies" and we always favor/send the latter. (Especially because constraint name to source dataset can be a 1:many relationship!)

aha! I knew I'd seen it somewhere in the code! I've switched it to use Entity type!

ok I ran into a clash with this - PlanningConstraint is in shared and prototypeApplication so I've added a weird temporary one for now until we understand what the sentiment comments are doing with the constraints.

jessicamcinchak · 2025-02-11T13:36:41Z

tests/usage.test.ts

+  //   examples: getJSONExamples<PostSubmissionApplication>(
+  //     'postSubmissionApplication',
+  //   ),
+  // },


🔖 promise to come back to this soon & make test setup more flexible 🙂

jessicamcinchak · 2025-02-11T13:42:33Z

types/schemas/postSubmissionApplication/Metadata.ts

+   * This is the date this application was published to the public
+   */
+  publishedAt: DateTime;
+}


Metadata is an interesting one to think about !

Our original thinking of having "PlanXMetadata", rather than "ApplicationMetadata", was that the metadata is specific to how the application was completed within PlanX as a system. If an application were to come from another system, it would likely have similar and different aspects than what PlanX gathers - so we imaged it would be like AnotherSystemMetadata, which could then trigger slightly different data shapes if necessary.

I wonder if Metadata for PostSubmission at this stage should actually be BOPSMetadata?

PublishedAt is a funny one. I originally created publishedApplicationSchema and then I switched it round to be the PostSubmissionSchema so we could do the redacted versions later on.

PublishedAt is the date it was published to the public API in the publishedApplicationSchema but it could equally apply to the private API in the PostSubmissionSchema

We need a publishedAt date in the dpr for now and its not really a 'stage' in the same sense as the others but maybe we could move it to data.application ? I've updated the metadata to include the BaseMeta data as well so they both allow for PublishedAt.

jessicamcinchak · 2025-02-11T13:43:42Z

types/schemas/postSubmissionApplication/README.md

+
+## The Application Process
+
+```mermaid


🧜‍♀️ Appreciate a good README, thank you!! Nice motivation to add similar in other types/schemas/ subdirectories too

jessicamcinchak · 2025-02-11T13:45:39Z

types/schemas/postSubmissionApplication/data/Appeal.ts

+
+type AppealBase = {
+  /**
+   * Reason for appeal


nit: If you prefix this with @description Reason for appeal it'll get encoded in generated schema doc ! See available keywords here https://json-schema.org/understanding-json-schema/reference/annotations

(Also totally understandable/relatable if these are still intended as notes-to-self at this stage 😅)

thats handy to know! Currently we don't know that much about the appeals data so this may be a free text field - we'll have to wait and see for this one! 😅

jessicamcinchak · 2025-02-11T13:49:32Z

types/schemas/postSubmissionApplication/index.ts

+    specialist?: SpecialistComments;
+  };
+  submission: PrototypeApplication;
+  metadata: PostSubmissionMetadata;


I'm a little surprised to not see any mention of files at the root level (I see they can be associated with a comment). Are the original user-uploaded documents intentionally omitted/not necessary for a post-submission register-view of the application?

(Perhaps another argument for different metadata again, as PlanX metadata send along required/recommended/optional files info, but the files themselves have now been stripped!)

Honestly I think I took them out and forgot to put them back in! 😂
We've got files all over the place now ones from submission, ones that are added to comments, some are added during appeal and theres also possibly versioning - so the nice neat files at the root level may not work so well anymore 🤔

apricot13 force-pushed the hn/published-application-schema branch from 1c1e4f1 to 114145b Compare February 6, 2025 10:07

apricot13 mentioned this pull request Feb 6, 2025

Add decisions (and consultations) into the ODP schema #276

Open

apricot13 changed the title ~~Published application schema~~ Post submission application schema Feb 6, 2025

apricot13 force-pushed the hn/published-application-schema branch from df4862c to 53f36e8 Compare February 10, 2025 14:49

jessicamcinchak reviewed Feb 11, 2025

View reviewed changes

apricot13 force-pushed the hn/published-application-schema branch 3 times, most recently from 1a95340 to bb4bf91 Compare February 18, 2025 18:01

Post submission schema

cff81fb

apricot13 force-pushed the hn/published-application-schema branch from bb4bf91 to cff81fb Compare February 18, 2025 19:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Post submission application schema #306

Post submission application schema #306

apricot13 commented Feb 5, 2025 •

edited

Loading

jessicamcinchak left a comment •

edited

Loading

jessicamcinchak Feb 11, 2025 •

edited

Loading

apricot13 Feb 18, 2025

jessicamcinchak Feb 11, 2025

apricot13 Feb 18, 2025

jessicamcinchak Feb 11, 2025

apricot13 Feb 18, 2025

apricot13 Feb 18, 2025

jessicamcinchak Feb 11, 2025

jessicamcinchak Feb 11, 2025 •

edited

Loading

apricot13 Feb 18, 2025

jessicamcinchak Feb 11, 2025

jessicamcinchak Feb 11, 2025

apricot13 Feb 18, 2025

jessicamcinchak Feb 11, 2025 •

edited

Loading

apricot13 Feb 18, 2025

Post submission application schema #306

Are you sure you want to change the base?

Post submission application schema #306

Conversation

apricot13 commented Feb 5, 2025 • edited Loading

Post Submission Application Schema

The Application Process

Stages

Status

How we all fit in

The structure

data.Application

data.localPlanningAuthority

data.submission

data.validation

data.consultation?

data.assessment?

data.appeal?

data.caseOfficer

Comments

metadata.publishedAt

Submission

Comments

Appendix

jessicamcinchak left a comment • edited Loading

Choose a reason for hiding this comment

jessicamcinchak Feb 11, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessicamcinchak Feb 11, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessicamcinchak Feb 11, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apricot13 commented Feb 5, 2025 •

edited

Loading

jessicamcinchak left a comment •

edited

Loading

jessicamcinchak Feb 11, 2025 •

edited

Loading

jessicamcinchak Feb 11, 2025 •

edited

Loading

jessicamcinchak Feb 11, 2025 •

edited

Loading