How Do You Evaluate an AI Tool Before Using It in a Therapy Practice?
Published: October 10, 2026
Artificial intelligence is making its way into therapy practices through electronic health records, documentation platforms, scheduling systems, and other software clinicians already use. Sometimes practice owners deliberately choose an AI product. Other times, a vendor introduces a new feature through a routine software update, and suddenly AI is available to staff who have been using that platform for years.
The convenience is appealing. AI can reduce administrative work, assist with documentation, and streamline certain practice operations. But before allowing a tool to access client information, someone needs to find out exactly what that technology does, where the information goes, and whether the vendor’s promises hold up under closer examination.
A vendor might advertise HIPAA compliance, offer a Business Associate Agreement (BAA), or assure customers that their information is secure. Those statements deserve attention, but they don’t answer every question a therapy practice needs to ask. A signed BAA doesn’t tell you whether every AI feature is covered, whether information is retained for other purposes, or what safeguards your practice must configure.
For behavioral health practices, those details matter. AI tools may process psychotherapy session recordings, clinical notes, diagnostic information, treatment plans, and other highly sensitive information. Before introducing that technology into clinical or administrative operations, practice owners need enough reliable information to determine whether the proposed use is appropriate.
Why Isn’t a Business Associate Agreement Enough?
A Business Associate Agreement is required when a vendor performs functions that make it a business associate under HIPAA. It establishes contractual obligations concerning protected health information (PHI), including permitted uses and disclosures, safeguards, and other responsibilities. However, signing a BAA doesn’t establish that every product or feature the company offers is covered.
Consider a group practice that has used the same electronic health record platform for several years. The vendor has a BAA with the practice, and everything has been running smoothly. Then the company introduces an AI assistant that can summarize client communications, draft progress notes, or analyze clinical information.
The practice already has a BAA, so it’s tempting to assume the new features are covered. But has anyone reviewed whether the agreement applies to those functions? Does the AI feature send information to another company for processing? Are the retention practices different from those associated with the original EHR?
An existing BAA may already cover the new functionality if its terms encompass the services and permitted uses involved. A separate agreement isn’t automatically necessary simply because a vendor introduces AI. What matters is whether the actual arrangement is covered by appropriate contractual protections and complies with the HIPAA Rules.
The U.S. Department of Health and Human Services (HHS) addresses the division of security responsibilities between vendors and customers in its Guidance on HIPAA and Cloud Computing. The guidance explains that covered entities and business associates need to understand their cloud service arrangements sufficiently to assess risks and establish appropriate safeguards.
A BAA is an important part of vendor oversight, but the agreement needs to be reviewed alongside the services being provided, the applicable terms, and the way the technology handles PHI. When vendors add optional AI capabilities, practices should confirm whether those capabilities fall within existing contractual arrangements or require additional review.
For practices that need assistance organizing business associate relationships and vendor oversight, Guardian Clinical Essentials™ offers a State-Specific Business Associate Agreement (BAA) Toolkit, which includes vendor evaluation and tracking resources.
Start With What the AI Tool Actually Does
Before getting buried in security certifications and vendor agreements, establish what the tool will be doing in your practice. AI is not one type of technology, and different functions can create very different concerns.
A scheduling assistant that helps organize appointments is not performing the same function as a platform that records psychotherapy sessions, generates clinical summaries, or recommends treatment interventions. Even within a single product, different features may access different information and rely on different processing systems.
For example, an AI documentation platform might offer transcription, progress note generation, diagnostic suggestions, and treatment planning assistance. A practice could determine that certain documentation functions are appropriate while deciding not to use features that generate clinical recommendations. That decision should reflect the actual technology, its intended use, and the clinician’s professional responsibilities.
Practice owners should be able to explain what information the tool requires, whether identifiable client information is necessary, who will have access to the system, and how its output will be reviewed. If a vendor cannot clearly explain what happens when a particular feature is activated, that is information the practice needs before proceeding.
Clinical applications also raise questions that go beyond HIPAA. An AI tool might have appropriate privacy and security safeguards while still producing inaccurate clinical information or recommendations that are unsuitable for a particular client. Protecting PHI and exercising professional clinical judgment are separate responsibilities.
Our article Can Therapists Use AI for Treatment Plans? examines those clinical considerations in greater detail.
What Happens to Client Information After AI Processes It?
This is one of the most important areas to investigate, and vendor marketing materials frequently leave out details that practice owners need.
Imagine using an AI documentation platform that records a therapy session and generates a progress note. The vendor advertises that recordings are automatically deleted after processing. That sounds reassuring, but what exactly gets deleted? Does the transcript remain available? Are portions of the information retained in processing logs? Are generated outputs stored separately? Does another service provider maintain copies?
The answers depend on the technology and the vendor’s arrangements. Different types of information may have different retention periods, and removing a recording from the user interface doesn’t necessarily mean every associated copy has been destroyed.
Practices should understand how information moves through the system from the moment it is submitted until it is deleted or returned. That includes identifying what is stored, how long it remains available, whether vendor personnel can access it, and what happens when the practice closes its account.
AI model training deserves particular attention. Some vendors explicitly prohibit the use of customer information for model training, while others have terms allowing certain uses for product development or improvement. Practices need to review the actual terms governing their service and determine whether any proposed uses of PHI are permitted.
A vendor’s promise that it doesn’t train AI models using client information is helpful, but it doesn’t establish that the vendor doesn’t retain or otherwise process the information. Those are different activities, and each deserves examination.
The same caution applies to claims that information is anonymous or de-identified. Removing a client’s name from a transcript doesn’t automatically make the remaining information de-identified under HIPAA. Clinical narratives can contain combinations of personal details that identify someone even when obvious identifiers have been removed.
HHS explains the recognized methods and requirements for de-identification in its Guidance Regarding Methods for De-identification of Protected Health Information.
When reviewing vendor claims, ask for explanations specific to the service and features your practice intends to use. General statements about privacy or secure processing are not enough to establish what happens to your clients’ information.
Who Else Is Handling Your Clients’ Information?
The company selling an AI product may not be the only company involved in processing information. AI platforms can rely on outside providers for cloud hosting, transcription, language models, infrastructure, and other services.
These arrangements are common in technology and aren’t automatically problematic. What matters is understanding whether additional companies create, receive, maintain, or transmit PHI and whether the required contractual protections are in place.
Under HIPAA, business associates generally must obtain appropriate written assurances from subcontractors that handle PHI on their behalf. HHS discusses these requirements in its Business Associates Guidance, including the responsibilities associated with downstream subcontractors.
A therapy practice ordinarily doesn’t need to negotiate separate BAAs with every subcontractor used by its business associate. However, that doesn’t mean the practice should ignore the vendor’s use of outside companies. The evaluation should consider who may process PHI, what functions those companies perform, and whether the vendor has appropriate arrangements governing that information.
Practices should also ask whether vendor employees can access identifiable client information for troubleshooting, quality assurance, or customer support. Access may be necessary for legitimate business purposes, but it should be governed by appropriate safeguards and limitations.
If a vendor cannot explain which other companies handle PHI or how those relationships are protected, the practice may not have enough information to approve the proposed use.
Your Vendor Handles Security. What Is Your Practice Responsible For?
A vendor may maintain strong security protections while the practice still has important responsibilities for how the product is configured and used.
Consider an EHR that introduces AI-generated clinical summaries. The platform may provide secure infrastructure, but the practice might control which staff members can activate the feature, view generated summaries, export information, or connect other applications.
If the practice enables the feature without reviewing permissions, employees could gain access to clinical information they don’t need for their responsibilities. The vendor’s security measures won’t necessarily prevent problems created by inappropriate account settings or poorly managed user access.
Depending on the technology, practices may need to address authentication, role-based permissions, audit capabilities, recording settings, account termination procedures, and other safeguards. The vendor should explain which protections are built into the service and which require action by the customer.
The HIPAA Security Rule requires covered entities and business associates to implement appropriate administrative, physical, and technical safeguards for electronic PHI. Those obligations don’t disappear when a practice purchases software from a company that advertises strong security.
Practices also need to understand whether an AI feature can be disabled, restricted to certain users, or configured to limit the information it processes. These options can affect whether a particular feature is appropriate for the organization.
When Should You Question an AI Vendor’s Claims?
Vendor evaluation shouldn’t feel like a game of twenty questions where every answer leads to another vague promise about security. A company that wants access to sensitive client information should be prepared to explain how its product handles that information and what protections apply.
One claim worth examining is that a product is “HIPAA certified.” HHS does not operate a general certification program declaring commercial software products HIPAA compliant. Independent security certifications and third-party assessments can provide useful information, but practice owners need to understand what was evaluated and whether the assessment applies to the product and features they intend to use.
HHS explains that it does not recognize private Security Rule certifications as proof of compliance in its Guidance on Certification and the HIPAA Security Rule.
Other situations warrant additional questions. A vendor might offer a BAA but be unable to identify which AI features are covered. Its website might promise immediate deletion while its service terms describe longer retention periods. A representative might insist that information is never used for model training but be unable to explain what the company’s product improvement provisions allow.
Conflicting answers deserve attention, particularly when the technology will process identifiable clinical information.
This doesn’t mean every unanswered question makes a vendor unsuitable. Sometimes additional documentation or clarification resolves the concern. A practice may also be able to restrict certain functions or adjust configurations to address identified risks.
However, if the vendor cannot provide enough information to evaluate a material privacy or security concern, the practice should not treat that uncertainty as resolved simply because the product is convenient or widely used.
Should You Document Your AI Vendor Evaluation?
Yes, and the reason goes beyond keeping another form in your compliance files.
Suppose your practice begins using an AI documentation platform. Six months later, the vendor announces a change to its data processing practices, or your practice discovers that a feature operates differently than originally understood. How will you determine what information was available when the tool was approved, which safeguards were considered, and whether the change affects your earlier decision?
Without documentation, you may be left trying to reconstruct conversations with sales representatives, locate old emails, and remember which product features were enabled at the time.
HIPAA doesn’t require every practice to complete a particular AI vendor evaluation form. The Security Rule does, however, establish requirements for risk analysis, risk management, and documentation of specified security activities. When technology affects electronic PHI, the associated risks need to be addressed within those processes.
HHS explains in its Guidance on Risk Analysis that organizations must assess potential risks and vulnerabilities affecting electronic PHI. The analysis needs to account for the organization’s systems and the ways information is created, received, maintained, and transmitted.
A useful vendor evaluation record identifies the technology and features reviewed, the intended use, the information involved, and the vendor’s responses to relevant questions. It should also document concerns identified during the review, any safeguards or restrictions established, and the reasons supporting the final decision.
The practice’s security risk analysis should account for the technology where appropriate. Vendor evaluations and security risk analyses serve related purposes, but they are not interchangeable.
Guardian Clinical Essentials™ offers a Solo Practice Security Risk Analysis designed to help solo practitioners identify and document risks affecting electronic PHI as part of their HIPAA security compliance responsibilities.
What Happens When a Vendor Adds New AI Features?
Technology changes, and a decision made when a product was originally purchased may not address everything that product does today.
A practice might approve a scheduling platform that doesn’t access clinical documentation. Later, the vendor introduces AI-generated communication summaries or other features that expand the information being processed. If staff begin using those capabilities without review, the practice may have introduced new risks without recognizing them.
This is particularly relevant when AI features are added to existing subscriptions. Practice owners may receive a product announcement, overlook an updated setting, or assume the new functionality is covered by the agreements already in place.
The HIPAA Security Rule includes a requirement for periodic technical and nontechnical evaluations, including evaluations in response to environmental or operational changes affecting the security of electronic PHI.
Not every minor software update requires an entirely new vendor review. The important consideration is whether a change affects how PHI is accessed, processed, stored, disclosed, or protected, or otherwise changes the risks previously evaluated.
Practices should have a process for reviewing relevant vendor announcements, changes to contractual terms, and newly available functionality. When those changes affect an earlier approval, the evaluation and supporting documentation may need to be updated.
THE AI FEATURE YOU DIDN’T KNOW YOU HAD
Your EHR vendor adds a new AI feature. Your practice already has a BAA, and the feature is included in your subscription. Does that mean it’s ready to use with client information?
Not until you’ve confirmed what the feature does, how it handles PHI, and whether your existing agreements and safeguards cover its use.
A familiar vendor doesn’t automatically mean familiar risks.
Does the NIST AI Risk Management Framework Apply to Therapy Practices?
The National Institute of Standards and Technology (NIST) developed the AI Risk Management Framework to help organizations identify and manage risks associated with artificial intelligence. NIST also published a Generative AI Profile addressing additional concerns associated with generative AI systems.
These resources offer guidance on evaluating AI systems, understanding risks, and establishing organizational oversight. They address considerations such as reliability, security, privacy, transparency, and accountability, which can be relevant when a therapy practice is deciding whether to adopt an AI product.
The NIST AI Risk Management Framework is voluntary and does not independently create new HIPAA requirements for therapy practices.
The NIST Generative AI Profile provides additional guidance concerning generative AI technologies.
Practices can use these resources to inform their evaluation procedures while continuing to follow applicable HIPAA requirements, state laws, and professional obligations. The framework provides guidance for managing AI risks, but it does not replace the legal requirements governing client information.
Before You Give Another AI Tool Access to Client Information
AI can be useful in therapy practices, but convenience shouldn’t be the deciding factor when selecting technology that will handle sensitive client information. A vendor’s assurances are a starting point for asking questions, not the end of the evaluation.
Practice owners need to know what a tool does, what information it accesses, who handles that information, how it is protected, and what responsibilities remain with the practice. They also need a record of the decisions made and a process for addressing changes after the technology has been introduced.
These responsibilities belong within the practice’s existing HIPAA compliance program. Vendor oversight, security risk analysis, workforce access, policies, and incident response procedures all contribute to protecting PHI when new technology enters the practice.
Guardian Clinical Essentials™ helps behavioral health practices examine technology-related compliance concerns, evaluate operational safeguards, and strengthen the policies and documentation supporting their HIPAA compliance programs.
For assistance reviewing your practice’s compliance responsibilities, explore HIPAA Compliance Consulting for Therapy Practices.
We’ll protect your practice so you can keep changing lives.
Guardian Clinical Essentials Resources
Guardian Clinical Essentials develops HIPAA, privacy, technology, and compliance resources specifically for therapists and behavioral health practices. GCE also monitors federal and state developments that may affect behavioral health practice and reviews existing resources when relevant requirements change.
For practical compliance resources, articles, tools, and free resources, visit Guardian Clinical Essentials.
This article is provided for educational and informational purposes and does not constitute legal advice. Laws, regulations, licensing requirements, and professional obligations may change. Mental health professionals should review the requirements applicable to their practice and consult qualified legal counsel when needed.
Frequently Asked Questions
About Evaluating AI Tools for Therapy Practices
Can I test an AI documentation tool before deciding whether to use it with clients?
Yes, therapists can test AI documentation tools using fictional client information before approving them for clinical use.
Invented session scenarios can help you evaluate the quality of generated notes, identify inaccuracies, and determine whether the tool fits your documentation workflow. Avoid using actual client recordings, clinical notes, or identifiable details during preliminary testing. A successful test can help you decide whether the product is worth pursuing, but the vendor’s handling of PHI still needs to be evaluated before introducing actual client information.
Can I approve an AI tool for administrative tasks but prohibit its use with client information?
Yes, a therapy practice can approve an AI tool for general administrative work while prohibiting staff from entering protected health information.
For example, the tool might be permitted for developing general staff training materials or organizing nonconfidential projects without being approved for clinical documentation. The practice should clearly identify which uses are permitted and communicate those boundaries to employees. Staff also need to understand that removing a client’s name doesn’t necessarily make clinical information safe to enter into an unapproved system.
What should I do if an AI vendor won't explain how long it keeps client information?
A therapy practice should not approve an AI tool to process PHI when it lacks the information needed to evaluate material data retention risks.
Ask the vendor for its retention and deletion terms, including whether transcripts, recordings, processing logs, or backups are retained. If the vendor’s answers are incomplete, request written clarification before making a decision. You may find that certain features can be disabled or configured differently, but unresolved concerns about how PHI is handled should not be ignored.
Can I limit which employees are allowed to use an approved AI tool?
Yes, group therapy practices can restrict approved AI tools to specific employees, roles, or functions.
A practice might authorize certain clinicians to use an AI documentation feature while prohibiting administrative employees from accessing it. These restrictions should reflect job responsibilities, the information involved, and the access controls available within the platform. Approval to use one AI feature also doesn’t give employees permission to activate every other feature the vendor offers.
What should I do if a vendor changes its AI features after my practice has approved the tool?
A therapy practice should review vendor changes that affect how an approved AI tool accesses, processes, stores, or shares client information.
A new feature might introduce additional processing services, different retention practices, or permissions that weren’t part of the original evaluation. Review the vendor’s updated documentation and determine whether existing agreements and safeguards still address the intended use. If the change introduces unresolved concerns, the practice can restrict or disable the affected functionality while completing its review.
Can I ask an AI vendor to delete client information after my practice stops using the service?
Yes, therapy practices can request deletion or return of client information when ending an AI vendor relationship, subject to the applicable agreement and legal requirements.
A BAA generally must address the return or destruction of PHI at termination when feasible, including information held by subcontractors. Practices should determine what the vendor can delete, whether information remains in backups, and what protections apply to any PHI that cannot feasibly be returned or destroyed. These details are worth reviewing before purchasing the service, rather than discovering the limitations when trying to close the account.
Continue Exploring AI and HIPAA
This article is part of the Guardian Clinical Essentials™ AI and HIPAA educational series. Visit the AI + HIPAA Resource Hub for Therapists for additional articles addressing AI documentation, privacy, security risk analysis, business associate agreements, and the use of artificial intelligence in behavioral health practice.
About the Author
Samantha Schalk, LMSW-C, LMSW-M, CAADC, CIMHP, BCP3, CHPSE
Samantha is a licensed mental health professional, private and group practice owner, Certified HIPAA Privacy Security Expert (CHPSE), and the founder of Guardian Clinical Essentials™.
Her work focuses on helping therapists and behavioral health practices understand and implement requirements involving HIPAA, privacy, security, documentation, technology, and practice operations.
Drawing from her experience in clinical practice, supervision, practice ownership, and compliance consulting, Samantha develops practical compliance resources, training, and implementation guidance specifically for behavioral health professionals.
Learn more about Samantha and Guardian Clinical Essentials™.
Continue Exploring Guardian Clinical Essentials™
Looking for additional compliance, privacy, AI, documentation, and practice operations resources?
Explore the Guardian Clinical Essentials™ Resource Library for educational articles, implementation guidance, training opportunities, and practical resources designed specifically for mental health professionals.
