Children’s Hospital of Philadelphia supported new statewide efforts to increase oversight of artificial intelligence tools and prevent AI-powered chatbots from presenting themselves as licensed healthcare professionals.
Here are eight things to know:
- Gov. Josh Shapiro on Feb. 27 announced the launch of a multiagency AI Literacy Toolkit and the creation of a 12-member AI Enforcement Task Force within the Pennsylvania Department of State. The measures are aimed in part at addressing concerns about AI companion bots that could provide misleading or inappropriate health guidance.
- The Children’s Hospital of Philadelphia reviewed and supported the state’s new AI guidance, according to a Feb. 27 news release from the governor’s office.
- “As the use of AI grows, we’re learning both the promise and the risks it poses to young people, who are especially vulnerable to misuse,” Tami Benton, MD, psychiatrist-in-chief at Children’s Hospital of Philadelphia, said in the release. “Children and adolescents can’t always tell helpful chatbots from sources of harmful mental health guidance, so all of us who are involved in their lives must stay vigilant.”
- The AI Literacy Toolkit was developed in collaboration with multiple state agencies, including the departments of Education, Health, Human Services, State, Aging and Banking and Securities. It expands existing digital literacy resources and includes information for parents discussing AI companion bots with children, classroom guidance for educators, warnings about financial scams and a link to a new complaint form for reporting AI-powered bots that may be engaging in unlicensed professional practice.
- State officials said the Department of State’s new AI Enforcement Task Force will evaluate complaints submitted through a dedicated form, hotline and email address to determine whether certain AI technologies are violating existing professional licensure laws.
- Secretary of the Commonwealth Al Schmidt said in the release that AI chatbots cannot legally present themselves as licensed healthcare professionals or provide mental or physical healthcare to Pennsylvanians.
- The Shapiro administration is also coordinating with Attorney General Dave Sunday to investigate potential consumer protection violations under the state’s Unfair Trade Practices and Consumer Protection Law, the release said.
- Mr. Shapiro first committed to addressing AI companion bots during his 2026-27 budget address. His proposed reforms would require age verification and parental consent for AI companion bots, mandate that companies direct children who mention self-harm or violence to appropriate authorities, require periodic reminders that users are interacting with AI rather than a human and prohibit sexually explicit or violent content involving minors generated by such bots.
In addition to Children’s Hospital of Philadelphia, the guidance was reviewed and supported by the Pennsylvania Chapter of the American Academy of Pediatrics, the Pennsylvania Chapter of the American Academy of Family Physicians and the Pennsylvania Medical Society, according to the release.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
