There might be another listener in the room at your doctor's appointments now or in the near future: ambient AI.

Instead of frantically typing notes during your next visit or updating medical charts into the wee hours of the morning, doctors at hospitals like Tampa General Hospital are using ambient AI to transcribe and organize patient notes in real time, says Dr. Nishit Patel, the vice president and chief medical informatics officer at TGH.

And other hospitals are doing the same.

"For every hour any of us as a provider, a physician, spends with the patient, we spend two hours at the keyboard trying to capture all of it," says Patel, who is also a practicing physician that sees patients several days a week. In his role at TGH, Patel is focusing on technology infrastructure and AI strategy.

Ambient AI is a technology that records and automates transcripts of conversations in real time — like a scribe. It allows doctors like Patel to "walk out with not a transcript, but an actual thoughtful, structured note shell that then I can tweak and edit around and go from there."

About 300 physicians and providers in the Tampa General Hospital system were able to try the technology and Patel says they found "a subset of those physicians and providers adopt it."

Over 15,000 patient visits were conducted "using this type of ambient AI technology to augment the documentation," he says.

"If we can mitigate the risks thoughtfully, this will be one of the most important, transformative moments from a healthcare delivery perspective."

Doctors will need to acknowledge patients' fears around AI

Dr. LaTasha Seliby Perkins, a family physician at Georgetown University, recently started training to use AI to help with her charting and plans to ask her patients for permission before recording appointments.

"I take care of baby boomers and the underserved, so 90% of my patient practice is Medicaid and Medicare patients," Seliby Perkins says. "There is some mistrust in the medical system, as it should be for a lot of Black and brown patients, right? And so when you introduce something new, you have to definitely be mindful of that."

It's a practice that Tampa General Hospital physicians follow when using ambient AI as well, Patel notes.

Seliby Perkins is prepared for patients to either decline the use of ambient AI during their visits or have a lot of questions about how it works.

How does she suggest responding to patients' concerns? "Just be very authentic with your patients about AI, your fear, acknowledge theirs, and be willing to answer any questions they have."

Maintaining the patient-doctor relationship is something Seliby Perkins is adamant about, despite the introduction of new AI technologies in healthcare. "Continuity of care and relationship building is important in a primary care setting. So anything that would reduce that in any way would be something I would have second thoughts about doing," she says.

Surveys polling patients' opinions on AI in healthcare spaces show "they're fearful of AI. They think that AI is going to take the physician out of the loop, and that an algorithm is going to decide [their] fate," Patel tells Make It. "And I don't think anything could be further from the truth."

Prior to implementing ambient AI at TGH, the hospital set up an AI governance group with physicians, ethicists and experts in compliance, risk, privacy and security.

"We want to make sure that anything we deploy that's in any way touching a patient," Patel says, whether directly or indirectly for operational efficiency, "that we're really thinking about that whole picture, not only the opportunity, but the risks that might be associated, any biases that might be introduced or exacerbated."

'It does have some hallucinations'

Some doctors like Dr. Nathaniel DeNicola haven't incorporated ambient AI in their practice. DeNicola is a board-certified OB-GYN in southern California who runs his own private practice; he also became the chair of telehealth work for the American College of Obstetricians and Gynecologists in 2016.

"We do not use [AI] to create patient notes," DeNicola says. "All of our notes are done by our clinicians or by our medical assistants, start to finish." His practice has used scribes before, but only humans.

"In exploring companies, there were ones that used AI. We chose not to use those. Not for dissatisfaction with the idea of AI, [but] the overall package wasn't what we're looking for," DeNicola says.

Still, AI shows up in DeNicola's non-clinical work. He uses AI to optimize his emails and call out which messages require his immediate attention, and generative AI helps him formulate the structure of some of his emails.

DeNicola has played with generative AI when researching for papers he's written. "If you wanted to see what a quick narrative overview of a topic is, it's pretty useful for that. It summarizes things rather well, and then it can be like idea generating for papers you're writing," he says.

But AI has its limitations. "It does have hallucinations," DeNicola says. "In my experience, it seems to make up references sometimes. Like it'll say something about a study, and then I go to check it, and the study doesn't exist."

Seliby Perkins also worries about the potential inaccuracies of AI and strongly urges professionals to double-check what it generates. Cultural differences like a patient's accent or dialect can be completely overlooked or misinterpreted by an AI transcription service.

"The reason why it's important for a doctor to go back and edit it is because even with like gathering statistics or gathering subjective kind of information from AI, it may not take into account, like, age, gender, cultural background when it comes to catching just tone and inflection," Seliby Perkins says.

"Where AI can help with some of the generalized things, to mitigate bias, we still have to keep in mind that there are going to be equity, inclusion and diversity issues that we have to remain mindful of when we're using artificial intelligence, because of the way it collects information."

Want to be a successful, confident communicator? Take CNBC's online course Become an Effective Communicator: Master Public Speaking. We'll teach you how to speak clearly and confidently, calm your nerves, what to say and not say, and body language techniques to make a great first impression. Get started today.

Plus, sign up for CNBC Make It's newsletter to get tips and tricks for success at work, with money and in life.