AI-assisted workflows

When an Audit Finding Does Not Match the System: What I Check First

A review of my bilingual website changed how I handled existing FAQ structured data, similar writing packages and article sharing images.

Huiyang Xie

On this page
    1. An existing FAQ was being treated as new work
    2. A similar package name can send the review in the wrong direction
    3. The OG template existed; image generation was a separate question
    4. How I turn a finding into a decision
    5. What this review established
  1. FAQ

A review report can make a project look less finished than it is. It can also make a real gap look smaller than it is. I encountered both possibilities while reviewing the system behind my personal website.

Some findings concerned capabilities that were already implemented. Others pointed to genuine uncertainty, but described it at the wrong level. An image template, for example, existed even though the handoff for producing a new article’s image still needed clarification.

I could have turned every finding into an implementation task. Instead, I asked for the conclusions to be checked against the actual system. The useful work began when we could say what each piece of evidence established, and what it left unresolved.

An existing FAQ was being treated as new work

One proposed change concerned the structured data for the website’s Skills FAQ. The page already displayed questions and answers. More importantly, the implementation already passed those FAQ items to the structured-data helper.

That distinction mattered. Seeing a FAQ on a page would not, by itself, establish that the corresponding structured data existed. The code gave us a more specific observation: the visible answers and the schema output were connected to the same source.

The immediate decision therefore changed. I asked for existing behaviour to be re-verified and tests added where needed, rather than for the feature to be implemented again. I also asked that the work log distinguish new changes from verification of existing work.

There was still a legitimate check to do. Existing code does not settle whether the intended version is deployed, whether output contains duplicates, or whether the visible text and structured data agree in the rendered page. Those are narrower questions than “Is this capability missing?”

If the verification had shown a mismatch, that mismatch would have become the repair task. Until then, treating the whole feature as absent would have made the report less accurate and the implementation scope larger.

This is the first distinction I want from an audit: does “missing” mean absent from the system, absent from the material inspected, or not yet verified?

A similar package name can send the review in the wrong direction

The writing-workflow review exposed a different problem. A general-purpose adaptation appeared in the search before the production package used for my personal site.

The two were related, which made the confusion plausible. Both dealt with English drafting, Chinese adaptation and editorial review. They did not have the same responsibilities. The general package deliberately left site-specific implementation and publication outside its scope.

Reviewing that adaptation against the personal site’s operating requirements could produce a tidy list of gaps while answering the wrong question. A responsibility omitted from the adaptation was not automatically missing from the production system.

We located the site-specific production source and reviewed that instead. The correction was about selecting the right object before judging it.

For a reusable tool or workflow, I want the review to establish:

  • The package’s intended purpose and scope.
  • Its version and provenance.
  • Whether it is a production source, an adaptation, an example or a backup.
  • Which copy is actually enabled in the environment being discussed.

The last point needs separate evidence. Finding a newer package in a repository does not establish that a running environment uses it. Similarly, a successful test of one copy cannot simply be assigned to another copy because their names look alike.

I do not need every version detail in an executive summary. I do need the reviewer to have resolved those details before recommending changes.

The OG template existed; image generation was a separate question

Open Graph, or OG, images are the images a site supplies for link-sharing previews. During the same review, the image requirement initially sounded as though the writing workflow needed an additional image-design capability.

I remembered that the website architecture already included an article OG template and asked for it to be checked.

The recheck confirmed that the template existed and used article fields to produce the established visual layout. The website’s delivery approach used saved PNG files. What was not established by the template alone was how each new article would get its final image generated, stored and checked.

The build configuration inspected at that point did not establish automatic export for every new article. That was a finding about the reviewed version and evidence, not a permanent statement that automation was impossible or would never be added.

The better diagnosis separated several questions:

  • Is there an approved layout for the image?
  • Are the article fields needed by that layout available?
  • Has a usable image file been produced for this article and language?
  • Does the page reference that file correctly?
  • Has the resulting output been checked?

A template answers the first question and helps with the second. It cannot answer all the others.

That changed the proposed remedy. We did not need a second visual design system inside the writer. We needed to clarify the handoff to the website and verify the resulting images before publication.

The same reasoning applies beyond images. A function can exist without being called. A file can exist without being included in a release. A release can complete without every user-facing result having been checked. The review has to follow the particular claim far enough to find the relevant evidence.

How I turn a finding into a decision

These examples gave me a practical review sequence. I use it before approving a change, especially when a report conflicts with something I remember being completed.

1. Make the claim specific enough to test

“FAQ needs work” leaves too much open. It might refer to missing answers, absent structured data, inconsistent text or an untested implementation.

I ask what should be present, where it should be present, and what was observed instead. If the reviewer cannot identify the inspected material, the finding needs investigation before it becomes a confirmed defect.

2. Fix the target version and environment

I identify the source revision or package, the page or artifact, the language, and the environment relevant to the claim.

A report about a production page should not quietly rely on a different preview. A conclusion about the enabled writing workflow should not rely solely on a related repository. When an exact deployment cannot be established, I keep that limitation visible.

3. Check the layer that can answer the question

Documentation explains intent. Code shows how a behaviour is implemented. A generated file provides evidence of an artifact. Rendered output shows what that particular execution exposes.

These sources can support each other. They still answer different questions. I avoid using one as a shortcut for a check that belongs elsewhere.

For the FAQ, implementation evidence changed the question from adding a feature to verifying its output. For OG images, template evidence prevented unnecessary redesign while leaving artifact-generation checks open.

4. Choose the smallest supported next action

Once the evidence is clear, I can assign the finding appropriately:

  • A reproduced defect needs a repair and a relevant recheck.
  • An existing implementation needs verification rather than duplicate construction.
  • A mismatch between documentation and behaviour needs reconciliation.
  • An unresolved observation needs a named check, with the unknown left explicit.

This keeps a plausible criticism from becoming an automatic permission to rewrite working parts of the system.

5. Preserve the correction in the record

The final report should say what changed and what was already there. Otherwise, the next review inherits a misleading history.

In the FAQ example, “verified the existing implementation and added checks” was a more accurate description of the requested work than “added FAQ structured data.” For the image example, the useful correction was that the website already owned the template while the production handoff needed clarification.

A short, accurate correction is enough. It should make the next decision easier without turning every review into a new documentation project.

What this review established

The review helped separate existing capabilities from checks that still needed to happen. It also showed that a detailed report could contain both a valid concern and an overbroad diagnosis.

I am not treating these incidents as evidence that the entire workflow is reliable, that all future audits will be correct, or that the website has gained search visibility. They are process observations from a specific review.

For me, the practical test of an audit is whether it supports the next action. I should be able to identify the target, the evidence, the remaining uncertainty and the reason for changing something.

The next time a report says a feature is missing, I will start there. That gives the implementation work a clearer scope and gives the completion record a firmer basis.

For the related question of when a verified stage can stop, see How to Define “Done” in an AI-Assisted Workflow.

Questions

Frequently asked questions

Should I ignore an audit finding if I remember the feature being built?

No. Treat that memory as a reason to locate the implementation and verify the relevant output. The feature may exist in another version, may not be deployed, or may not behave as intended. Memory helps direct the check; it does not replace it.

Is source code enough to prove that a feature works on the live website?

Source code can establish that an implementation exists in the inspected revision. A live-site claim also needs evidence about the deployed version and its relevant output. Keep those conclusions separate when either check is missing.

What should a useful audit finding contain?

It should identify the inspected target and version, the expected behaviour, the observed evidence, and the unresolved check or proposed change. The level of detail should be enough for someone else to reproduce the observation or understand its limits.

Does having an OG template mean each new article already has a sharing image?

No. The template defines a layout. The article still needs the required fields, a generated image artifact, the correct page reference and appropriate checks under the site’s actual delivery process.

What if the report and the implementation disagree?

Recheck the report’s target and the layer it inspected. If the discrepancy remains, keep both observations and investigate the specific gap. Correct the finding when evidence disproves it; repair the implementation when the defect is reproduced. Do not force agreement by changing the system first.

Share

Found this useful? Share it.

LinkedInXCopy link

Discuss

Want to discuss the idea?

Discuss with me

Author

About the author

Huiyang Xie is a marketing professional based in Greater Vancouver, Canada, working across digital marketing, content, websites, AI-assisted workflows, and cross-cultural marketing. Her work explores how AI can support practical marketing processes while keeping factual review, brand judgment, and human approval in the loop.

Continue reading

Related work and reading

Blog article

How to Define “Done” in an AI-Assisted Workflow

Open