Artificial intelligence (AI) is rapidly reshaping the American landscape, transforming how individuals and organizations operate across virtually every sector. From corporate boardrooms to family kitchens, AI-powered tools are being adopted to streamline workflows, automate routine tasks, and unlock insights from vast amounts of data.
Industry leaders such as OpenAI, Google, and Anthropic now serve millions of users each month, while a growing ecosystem of technology companies is racing to develop specialized AI solutions tailored to specific industries and business functions. Most of these applications are built upon foundational large language models, extending their capabilities to address broadly used and unique operational needs alike.
Despite its remarkable advancements, AI is not yet a substitute for human judgment. Rather, it is best viewed as an intelligent collaborator—a powerful thinking partner that enhances decision-making and productivity.
Organizations seeking to capitalize on AI’s potential should do so thoughtfully, implementing appropriate oversight, validation, and governance measures to maximize value while mitigating risk.
Two risks should remain top of mind when considering an AI solution for business use: privacy and hallucination. Sensitive information should never be inputted into an off-the-shelf AI solution. While enterprise AI solutions may provide heightened data protection as part of the subscription or through separately negotiated agreements (e.g., a Data Processing Agreement), off-the-shelf AI solutions do not ordinarily offer the same protection. In fact, several leading AI solutions expressly provide that user input (and in many cases outputs) can be used to train the AI model. In short, information entered into the AI solution could be retained, analyzed, and even publicly disclosed by the developer of the tool. An affirmative opt-out is often required to disable the use of user inputs for training. AI solutions also continue to hallucinate. Countless examples across industries continually emerge, demonstrating the risks of AI hallucinations.
One survey[1] found that 62% of adults in the United States indicated they interact with AI at least several times a week. Another survey identified that 26% of employees in the U.S. used AI daily or a few times a week in the workplace. With the misuse of AI continuing to make headlines across industries, executives and business owners alike are right to ask: how might we implement AI in our business without inadvertently exposing our data or forfeiting certain privacy protections? Although the precise answer requires a thoughtful analysis of the business vertical, the sensitivity of the data, and the desired AI solution(s), the following fundamental concepts can guide the use of AI in a business context.
Key Considerations when Leveraging AI in Business
- Prompt Thoughtfully: Never input confidential, privileged, or sensitive data into an off-the-shelf AI solution. Doing so may inadvertently waive confidentiality protections or legal privilege. If there is a business case to input sensitive data into an AI solution, consider an enterprise solution and obtain legal review of any subscription terms.
- Verify all Outputs: Any AI output used in a professional or regulated context requires independent human verification. Unvetted AI outputs can harm the business and consumers alike and invite legal and regulatory scrutiny.
- Existing Laws Apply: Unless superseded, existing laws and regulations may govern a business’ use of AI (e.g., consumer protection laws). Regulated entities, such as those in the insurance industry, must continue to comply with applicable law. Regulators may use existing authority to inspect AI usage and existing laws to impose penalties for improper AI usage.
- Build a Strong AI Foundation: Develop a written AI governance program. Implement routine audits to evaluate processes and check for bias. Memorialize vendor arrangements through a written agreement. Involve legal counsel in the review of vendor agreements and prior to using off-the-shelf AI solutions for any sensitive or proprietary tasks.
|
The Use of AI in Insurance is on the Rise
Our Fall 2025 article surveyed the rise of AI regulation in the insurance industry, chiefly through the widespread adoption of the NAIC Model Bulletin on AI. Several state-specific AI laws took effect this year, some of which directly impact the business of insurance.
Companies operating in regulated industries such as the business of insurance face heightened regulatory scrutiny and reporting obligations. Pair this fact with the propensity for insurance industry members to obtain, process, and retain personally identifiable information from consumers, and the use of AI in the insurance industry presents ample risk. Notwithstanding these risks, adoption of AI within the insurance industry has accelerated substantially.
Our December 2025 article explored the implications of a generative AI platform rendering insurance coverage advice. But what if a regulated insurance entity employed an AI solution in connection with the business of insurance? This is now the reality. One survey found that 84% of health insurers use AI or machine learning in their operations, and 76% of insurance firms have deployed generative AI in at least one business function. From underwriting and pricing to claims and customer service, AI is being deployed across the full insurance lifecycle.
Consider, for example, the following hypothetical: an independent insurance agency licenses a third-party AI solution marketed as a “quoting assistant” to improve customer service and increase efficiency by leveraging an AI chatbot to quote insurance coverage. Licensed producers at the agency begin relying on the coverage recommendations without independent review or validation. The agency does not develop a written AI governance policy, nor does it disclose to consumers that they are interacting with AI. The AI tool routinely recommends inadequate coverage and improperly uses zip code location to steer certain consumers to higher-premium products without sufficient actuarial justification. Consumers lodge complaints with the state insurance department and a regulatory inquiry follows.[2]
The upshot? The use of AI in insurance, without independent review and validation, poses serious regulatory risk. Licensed insurance entities and individuals cannot outsource their licensed duties to an AI solution. Liability rests with the licensed insurance producer or broker agent. Insurance entities deploying AI tools must carefully vet the tool, establish appropriate data privacy regimes, train staff, disclose AI use when required or advisable, and retain ultimate human control and oversight. Significant decisions relating to underwriting, claims processing, and other material processes cannot be fully delegated to AI. Members of the insurance industry that are considering leveraging an AI solution should collaborate closely with legal counsel to ensure compliance with all applicable laws.
What This Means for You
Whether an organization is evaluating its first AI solution or expanding existing AI capabilities, proactive governance and legal review can help maximize value while minimizing risk. Husch Blackwell’s Insurance Regulatory attorneys work with clients to navigate emerging technologies and the evolving requirements that accompany them. To discuss your organization’s AI strategy or compliance obligations, please contact a member of our team.
Contact Us
If you have any questions about Deploying AI as a Tool for Your Insurance Business please contact Lauren Ybarra, Tasha Cycholl, Jared Bruttig, or your Husch Blackwell attorney.
[2] Potentially implicating unfair trade practices statutes and state anti-discrimination laws.