AI-Written Custom Software: What Your Studio Contract Should Require

The buying decision: require evidence-backed accountability, not a vague AI promise

A total AI ban can sound safer than it is. It may be difficult to verify, it does not make human-written code secure, and it can push AI use into an informal workflow where there is less oversight. A broad permission such as “the contractor may use generative AI” has the opposite problem: it says nothing about accounts, data handling, review, ownership, security, or evidence.

The stronger position is conditional permission. Allow AI where it fits the sensitivity of the component, restrict it where the consequences are higher, and prohibit it where an external obligation or the studio’s inability to provide evidence makes use unacceptable.

The contract should regulate the delivered result and the process risks that affect that result. At minimum, it should prevent the studio from using AI as a disclaimer. The studio remains responsible for meeting the specification, protecting confidential information, complying with licenses, reviewing the code, correcting defects, and completing the handoff.

That leads to one practical hiring decision: **hire a studio using AI coding tools only if it will accept responsibility independent of how the code was produced and will deliver verifiable evidence of that responsibility.**

A component-level decision tree: permit, restrict, or prohibit

Do not assign one AI policy to the entire project. A marketing-page component, an internal reporting screen, an authentication service, and a proprietary pricing engine do not present the same consequences. Classify components during scoping and record the decision in an AI-use schedule attached to the agreement.

1. **Does a law, customer agreement, data-use restriction, license, or internal policy prohibit the proposed AI use?** If yes, prohibit AI assistance for that component or change the proposed tool and workflow until they comply. 2. **Would the tool receive confidential source code, credentials, personal information, customer records, production data, or proprietary logic?** If yes, either prohibit that transfer or restrict work to specifically approved accounts with agreed retention, training, access, and deletion settings. 3. **Could a defect materially affect authentication, authorization, payments, encryption, privacy, safety, financial calculations, production infrastructure, or other critical operations?** If yes, classify the component as restricted. Require named human reviewers, stronger testing, documented findings, and separate security acceptance. If the studio cannot produce that evidence, prohibit AI use for that component. 4. **Is the component lower-sensitivity and covered by ordinary engineering controls?** AI assistance can generally be permitted, but the baseline requirements for review, testing, licensing, repository history, and acceptance still apply.

| Decision | Appropriate when | Minimum contract position | |---|---|---| | **Permit** | No external restriction applies, approved tools can be used without exposing prohibited data, and normal review provides adequate control. | Use only registered tools and accounts; preserve repository history; complete human review, tests, license checks, and acceptance evidence. | | **Restrict** | The component processes sensitive data, controls important permissions, implements proprietary logic, or carries material security or operational consequences. | Limit tools, accounts, data, repository access, and agent authority; require designated reviewers, enhanced scans, documented risk decisions, and security acceptance. | | **Prohibit** | An external obligation forbids use, the tool’s data practices are unacceptable, or the studio cannot provide the required evidence and remedies. | No AI assistance for the named component; require the studio to attest to compliance and preserve evidence sufficient to investigate a suspected violation. |

“Restricted” should describe the actual restriction. It might mean that an assistant may suggest tests but may not access production data, that an agent may open a pull request but may not merge it, or that a tool may work within one repository but may not read credentials or deployment environments. Avoid labels without operational meaning.

Put the three questions into the contract’s operating structure

The agreement should connect permission, evidence, and remedies rather than scattering AI language into an isolated policy exhibit.

NIST’s Secure Software Development Framework provides buyers and producers with common language for documenting security requirements, sharing component provenance, and tracking security risks and design decisions. NIST has also demonstrated how those practices can be integrated into modern development pipelines through its live DevSecOps implementation guidance. These are useful reference points, but a reference to a framework does not replace project-specific deliverables.

CISA advises buyers to examine product security rather than relying only on organization-level compliance and to put appropriate product-security requirements into the contract. Its Secure by Demand guidance supports treating the software itself—not merely the vendor’s policy portfolio—as the object of acceptance.

A workable agreement package usually needs five connected parts:

- **An AI-use schedule** classifying components and listing approved tools, accounts, purposes, and restrictions. - **A security and data schedule** defining protected information, prohibited inputs, access limits, review requirements, scans, tests, and incident procedures. - **An evidence and acceptance schedule** naming the artifacts due at each milestone and the conditions for acceptance. - **An IP and software-component schedule** covering assignment, background materials, third-party components, licenses, and the SBOM. - **A remedies and handoff schedule** stating what happens after unauthorized use, missing evidence, failed tests, an IP issue, a security incident, or an incomplete transition.

Treat these as connected controls. For example, calling authentication code “restricted” accomplishes little unless the contract also names the required reviewer, the security tests, the evidence due, the acceptance consequence, and the correction obligation.

Requirement group 1: approved tools and accounts

A contract should not merely permit “commercially available AI tools.” The relevant risks depend on which service is used, under whose account, with what settings, and with access to which repositories and systems. NIST’s ICT supplier due-diligence guidance identifies provenance, foundational cybersecurity practices, supply-chain tiers, and ownership or control as supplier-assessment considerations.

| Business risk | Contract requirement | Evidence the studio should deliver | Red flags | |---|---|---|---| | Developers use personal or free accounts that lack the agreed organizational controls. | Require AI work to occur only through approved organizational accounts. Prohibit personal accounts for project code or data. Name who administers access and who can authorize exceptions. | Approved-tool register; account owner and administrator; permitted user list or role record; relevant configuration export or screenshot; access-removal record at offboarding. | “Everyone uses their preferred tools”; no central administrator; shared credentials; inability to identify which accounts accessed the repository. | | An AI tool gains broader repository, terminal, cloud, ticketing, or deployment access than the project requires. | Define the allowed integrations and least-privilege scope for each tool. State whether an agent may read files, edit code, execute commands, open pull requests, merge changes, or deploy. | Tool-permission record; repository integration settings; service-account scopes; branch-protection settings; access-review record. | The agent can merge or deploy without human approval; tool permissions are described as “standard access”; no one can produce a current access list. | | The studio changes models, vendors, plugins, or agent capabilities without evaluating the effect on data and security. | Require prior written approval for material changes to the approved-tool register. Define material changes to include a new provider, new data use, broader repository access, or autonomous execution authority. | Dated tool register; change request; risk review; buyer approval; updated data settings and access record. | The contract allows unrestricted substitutions; the studio claims all coding assistants are equivalent; changes leave no audit trail. |

Requirement group 2: confidential-data boundaries

AI coding tools may receive more than a pasted code fragment. Repository-aware assistants and agents can access source files, issue descriptions, logs, environment variables, terminal output, database schemas, and test fixtures. The FTC has warned that AI services can receive confidential business and customer information and that providers may face liability when they break commitments about data use, including commitments concerning model training. Its discussion of privacy and confidentiality commitments is a reason to document settings and permitted inputs instead of relying on assumptions.

| Business risk | Contract requirement | Evidence the studio should deliver | Red flags | |---|---|---|---| | Confidential source code, customer records, personal information, credentials, or production data is sent to an unapproved service. | Define prohibited inputs. State that protected information may be processed by an AI service only when the named service, account, purpose, and data treatment have been approved. Treat prompts, uploaded files, retrieved context, and tool output as part of the data flow. | Data-flow description; approved-tool register; model-training, retention, and sharing settings; prompt or activity records where required; secret-scan results. | The studio says it does not “paste data” while an agent has repository or terminal access; no one has reviewed the provider settings; production data is used casually for debugging. | | A provider retains project material, uses it for training, or allows human review contrary to the buyer’s requirements. | Specify the allowed retention period, training status, provider access, deletion process, and subprocessors where those issues matter to the project. Require notice before those conditions change. | Provider configuration; applicable service terms retained with the project record; deletion confirmation when available; material-change review. | Verbal assurances replace settings or terms; the studio cannot say whether project inputs are retained or used to improve a model. | | An unauthorized disclosure is discovered but handled as an informal development mistake rather than an incident. | Define unauthorized AI disclosure as a reportable event. Require containment, preservation of relevant records, prompt notice within a negotiated period, scope analysis, credential rotation when applicable, deletion efforts, and a written corrective-action record. | Incident-response plan; access and activity records; notification record; scope analysis; credential-rotation evidence; corrective-action report. | No AI-specific path into incident response; logs are unavailable; the studio decides unilaterally whether the buyer needs to know. |

Requirement group 3: disclosure and provenance

“We use AI” is too broad to support oversight, while an attempted inventory of every autocomplete suggestion may be expensive and unreliable. Define material AI use in terms that matter to the project: code or configuration incorporated into a deliverable, generated migrations, generated tests used for acceptance, agent-created pull requests, AI-assisted security decisions, or AI processing of protected information.

Provenance evidence should show how code entered the repository, who reviewed it, what tools were authorized, and what decisions were made. Repository history alone does not prove whether a human or model originated every line. An SBOM identifies components and dependencies; it does not establish line-by-line human authorship. Use several artifacts together.

| Business risk | Contract requirement | Evidence the studio should deliver | Red flags | |---|---|---|---| | The buyer cannot determine where material AI assistance affected the deliverable. | Require disclosure of material AI use by repository, component, or pull request. Define the required level of detail and how long records must be retained. | AI-use register linked to components or pull requests; approved-tool register; repository history; pull-request metadata. | Only a project-wide “AI was used” statement; refusal to identify affected components; disclosure occurs only after a problem. | | Large generated changes bypass the normal development trail. | Require development in the buyer-accessible repository, identifiable commits, pull requests for material changes, and preservation of meaningful history. Prohibit delivery solely as an unexplained archive or one final code dump. | Repository access; branch and commit history; pull requests; review comments; build records; release tags. | A single bulk import at the end; routinely deleted branches and comments; unexplained squashed changes; no buyer repository access until final payment. | | The studio records tool use but not the human decisions that make the output acceptable. | Require review records for designated components, including the reviewer, scope, tests considered, findings, exceptions, and approval. Require architecture or security decisions to be documented when they affect stated requirements. | Human-review record; pull-request approval; architecture decision record; risk-acceptance record; test linkage. | “A senior developer looked at it”; the generator and approver are the same automated agent; exceptions are closed without a rationale. |

Requirement group 4: IP assignment and license compliance

Paying for development does not, by itself, answer every ownership question. Under 17 U.S.C. § 204(a), a transfer of copyright ownership generally must be documented in a signed writing. The agreement should contain an express assignment of the rights the studio can transfer, obtain corresponding rights from personnel and subcontractors, and identify materials that are not being assigned.

AI introduces a separate copyrightability issue. The U.S. Copyright Office’s report on copyrightability and artificial intelligence explains that AI assistance does not prevent protection for human-authored expression, but purely AI-generated material and material lacking sufficient human control are not protected. Prompts alone generally do not provide the required control under currently available technology.

An assignment transfers available rights; it does not turn noncopyrightable material into copyrighted material. The commercial response is not to ask the studio to guarantee a legal conclusion it cannot establish. Require deliberate human authorship and review where exclusivity matters, disclosure of background and third-party materials, license compliance, and remedies if the delivered software cannot be used as promised.

| Business risk | Contract requirement | Evidence the studio should deliver | Red flags | |---|---|---|---| | The buyer assumes payment transfers ownership, but the agreement contains no effective assignment or does not cover contributors. | Include a signed, present assignment of applicable deliverable rights, subject to clearly listed exclusions. Require the studio to secure corresponding obligations from employees and subcontractors. Address further assurances and delivery of source materials. | Signed agreement; contributor or subcontractor certification; background-material schedule; final assignment confirmation if counsel requires it. | “You own it because you paid”; assignment language applies only after an undefined event; subcontractors are outside the IP obligations. | | AI output, copied snippets, or third-party packages create unknown license obligations. | Require an open-source and third-party component policy, license review, disclosure of obligations, and buyer approval for components that impose material distribution, source-disclosure, attribution, or usage conditions. | License report; dependency inventory; source and version data; approvals; notices file; SBOM. | No license scan; “open source means free to use”; undisclosed reciprocal obligations; dependencies without identifiable versions or sources. | | The deliverable depends on material the studio will not assign and the buyer cannot replace. | Require a background-material schedule before use, stating ownership, license terms, support implications, and replacement options. Prohibit unlisted proprietary dependencies without approval. | Background-material register; license copies or references; dependency map; approved exception; replacement plan for critical dependencies. | Background technology is defined so broadly that it includes most of the application; critical code is licensed only for the studio’s account; exclusions appear at handoff. |

Software-supply-chain guidance recommends using contractual mechanisms for security controls and identifies missing supply-chain requirements and an absent SBOM as procurement risks. See the joint recommended practices for software customers. Updated SBOM minimum elements include fields concerning author signature, component versions and hashes, identifiers, coverage, timestamps, and dependency relationships, with applicability to open-source, AI, SaaS, and other software types. The contract should identify the required SBOM format, delivery points, coverage, and update obligation rather than merely requesting “an SBOM.”

Requirement group 5: human review and security acceptance

Passing functional tests does not establish that software is secure. Research on code-agent patches found that functionally correct patches could still contain exploitable vulnerabilities. The study tested 12 agent-model combinations and, in one reported configuration and weakness category, found an attack success rate of 40.7%. See When Correct Is Not Safe.

A separate study evaluating more than 500,000 Python and Java samples found distinct defect profiles and reported more high-risk security vulnerabilities in the AI-generated samples. See Human-Written vs. AI-Generated Code. These findings do not mean every AI-generated line is unsafe or that human code is safe. They support treating functional correctness and security acceptance as different gates.

| Business risk | Contract requirement | Evidence the studio should deliver | Red flags | |---|---|---|---| | Generated code passes feature tests but introduces an exploitable weakness. | Define security acceptance separately from functional acceptance. Identify applicable threat scenarios, scan types, manual review requirements, severity thresholds, exception approval, remediation, and retesting. | Static-analysis results; dependency and secret scans; dynamic or application testing where appropriate; manual security-review record; finding dispositions; retest results. | “All tests passed” refers only to feature tests; scans are run but findings are not delivered; the studio decides exceptions without buyer approval. | | A developer accepts AI output without understanding the affected code. | Require qualified human review before merge. For restricted components, name reviewer qualifications or roles, require independence where appropriate, and require the reviewer to examine the complete change in context rather than only the generated fragment. | Pull-request approval; reviewer identity and role; review checklist; linked tests; comments and finding resolutions. | Automatic approval; review completed by the same agent that generated the change; reviewers cannot explain the code or its trust boundaries. | | Acceptance occurs before required security evidence exists. | Make delivery of the required evidence a condition of acceptance. State that time-lapse acceptance does not apply while required artifacts, fixes, or retests are outstanding. Connect milestone payment to agreed acceptance status. | Acceptance-test result; evidence index; open-findings register; exception approvals; signed milestone acceptance. | Security documents are promised after launch; production deployment is treated as automatic acceptance; missing evidence is classified as minor documentation work. |

Automated tools produce false positives and cannot establish that software is free of every vulnerability. The contract should therefore require findings to be reviewed and resolved, not demand a cosmetically empty report. A valid disposition might be remediation, evidence that the finding does not apply, or a documented buyer-approved exception with compensating controls.

Requirement group 6: liability, incident response, remedies, and handoff

A requirement without a consequence is often just a preference. Remedies should correspond to the business risk and be usable during the project—not only after expensive litigation. Counsel should tailor warranties, indemnities, liability caps, exclusions, insurance, and survival provisions to the specific transaction.

| Business risk | Contract requirement | Evidence the studio should deliver | Red flags | |---|---|---|---| | An unapproved AI tool or prohibited data flow is discovered. | Require immediate cessation and containment, disclosure of affected systems and data, preservation of records, deletion efforts, credential rotation where relevant, security review, and replacement or rewriting of affected work. State who pays for investigation, remediation, and retesting when the studio violated the agreement. | Tool and activity records; scope analysis; deletion request or confirmation; rotated-credential record; rewritten commits; security results; corrective-action report. | The only remedy is a promise not to do it again; the studio can withhold the name of the provider; investigation costs automatically fall on the buyer. | | Required provenance, review, SBOM, license, or test evidence is missing. | Make the artifact a condition of acceptance. Permit rejection or withholding of the associated acceptance until complete evidence is delivered and verified. Set a correction and resubmission process. | Evidence index; artifact delivery receipt; corrected submission; acceptance record. | Acceptance occurs automatically despite missing artifacts; documentation is due only after final payment; the studio can substitute a policy statement for project evidence. | | Delivered code infringes rights, violates an agreed license requirement, or cannot be used as represented. | Require correction, replacement, procurement of necessary rights, or refund where correction is not feasible. Have counsel address IP warranties, defense obligations, indemnity, exclusions, control of claims, and any applicable liability-cap treatment. | License report; rights documentation; replacement release; updated SBOM; notices; claim and remediation records if an issue occurs. | The studio disclaims all responsibility for AI-assisted code; the buyer’s sole remedy is to stop using the software; IP obligations exclude all third-party and AI-related material. | | A security defect, confidentiality incident, or incomplete handoff affects continued operation. | Define notice and cooperation duties, remediation and retesting, root-cause analysis where warranted, and survival after termination. Condition final acceptance or payment on delivery of the repository, build and deployment instructions, credentials under buyer control, documentation, SBOM, license report, test evidence, and open-risk register. | Incident plan; notification record; root-cause report; final repository history; reproducible build or deployment record; credential-transfer checklist; final SBOM; open-findings register; signed handoff. | A liability cap makes the promised remedy commercially meaningless; confidentiality and security obligations end at launch; the studio retains the only deployment credentials; “source code delivery” excludes build, configuration, or dependency information. |

Do not leave “incident” undefined. An unauthorized disclosure to an AI provider, a tool operating outside its permitted scope, exposure of a secret in a prompt, and an agent making an unapproved production change may require different responses, but each needs an agreed path for notice, containment, evidence preservation, and correction.

Handoff is also a remedy. If the relationship ends after a control failure, the buyer should not become more dependent on the studio that failed the requirement. The transition package should be sufficient for an authorized replacement team to inspect the current state, build it, deploy it, understand unresolved risks, and continue operating it.

The evidence checklist to attach to milestones

Ask the studio to identify the exact artifact, owner, delivery date, and acceptance consequence for each item. “Available on request” is weaker than scheduled delivery because the absence may not be discovered until a dispute or incident.

- [ ] **Approved-tool register:** provider, tool, account type, administrator, permitted purpose, allowed repositories, agent authority, relevant data settings, approval date, and material changes. - [ ] **Component AI classification:** permitted, restricted, or prohibited status for each sensitive component, with the reason and required controls. - [ ] **AI-use record:** material AI use linked to a component, pull request, or release at the agreed level of detail. - [ ] **Repository history:** buyer access, commits, branches, pull requests, reviewer identities, release tags, and build records. - [ ] **Human-review record:** reviewer, scope, tests, security considerations, findings, exceptions, and approval for restricted work. - [ ] **Security evidence:** static analysis, dependency scan, secret scan, relevant dynamic tests, manual review, finding dispositions, and retest results. - [ ] **Acceptance-test result:** functional and security outcomes shown separately, with unresolved exceptions identified. - [ ] **SBOM:** components, versions, identifiers, hashes where required, dependency relationships, coverage, timestamps, and authorship or signature fields required by the selected specification. - [ ] **License report:** detected licenses, obligations, notices, approved exceptions, and identified background materials. - [ ] **Rights record:** signed assignment, contributor and subcontractor coverage, and a schedule of excluded or licensed materials. - [ ] **Data-control evidence:** approved data flows, provider settings, access scopes, retention and deletion terms, and incident records if applicable. - [ ] **Handoff package:** current repository, build and deployment instructions, buyer-controlled credentials, operating documentation, final SBOM and license report, test evidence, and an open-risk register.

None of these artifacts is sufficient by itself. An approved-tool register does not prove code was reviewed. Repository history does not prove copyrightability. An SBOM does not show whether an AI agent generated a proprietary function. A clean automated scan does not prove the absence of vulnerabilities. Acceptance should be based on a coherent evidence set rather than one reassuring document.

Pre-signature checklist for the buyer

Before signing, confirm that the agreement answers these questions without requiring an oral explanation:

1. **Has each sensitive component been classified as AI-permitted, AI-restricted, or AI-prohibited?** 2. **Are approved tools, organizational accounts, integrations, and agent permissions identified?** 3. **Does the contract define which code, records, credentials, customer information, production data, and proprietary logic may not enter an AI service?** 4. **Must the studio obtain approval before materially changing providers, data practices, access, or agent authority?** 5. **Is material AI use disclosed at a useful component, pull-request, or release level?** 6. **Will the buyer have repository access and meaningful history during development rather than receiving a final archive?** 7. **Are human review, functional tests, and security acceptance separate, named obligations?** 8. **Are the SBOM, license report, scan results, review records, and acceptance results scheduled deliverables?** 9. **Does a signed writing assign applicable rights and identify background and third-party materials?** 10. **Do missing evidence, unauthorized tool use, license problems, security failures, and data disclosures have specific correction and response paths?** 11. **Are payment and acceptance tied to evidence, remediation, retesting, and handoff rather than delivery alone?** 12. **Can another authorized team inspect, build, deploy, and continue the software from the handoff package?** 13. **Has qualified counsel reviewed the IP, confidentiality, indemnity, liability, incident, and survival language for this project?**

If the studio objects to reasonable evidence because “AI is only a tool,” the answer is that tools affecting confidential data, source provenance, licensing, security, and acceptance belong in the production controls. If the studio claims AI creates no special risk, it should still be able to accept ordinary responsibility for the delivered software and show its work.