Hello everyone, this is Yao, and I'm very happy and honored to be invited here to give the talk about Data-Driven Machine Learning, Unlocking the Future of Closed-Loop Diabetes Care. So before I start introducing the projects, I would like to give a brief introduction on myself. So I pursued my PhD at UC San Diego on AI safety, so there is more like to how to make sure that all these deep learning-based models can be safely deployed in real world. And after that, I joined Google Brain as a research scientist, and mainly working on AI safety plus multi-model. So it's kind of like not only focusing on the image domain, we can also integrate the information from the language domain, like the image, text, and video. So the general goal is still that when we design the AI models, we hope that the AI can be safely deployed in the real world with multiple information sources. And from 2023, I started joining UC Santa Barbara as an assistant professor, and there I further expand my research interests from AI safety, multi-model AI, to AI for healthcare, and especially I'm super interested in AI for diabetes. So this is actually dated back to 2011, when I was diagnosed with type 1 diabetes. And as a patient, like right now I live with diabetes over 10 years, it's kind of like you know where patients are suffering from. And I'm super glad that combining my expertise in AI, because I have already done research in AI for many years, and then now it finally can converge, like using my AI expertise to help patients like me to better control our blood glucose and live freely as normal people. Hopefully this is a final goal that we want to achieve. And regarding the power of artificial intelligence, I believe that most people have already seen a breakthrough it has achieved in many different areas. For example, the JGPT from OpenAI, I think a lot of people are using it in your daily life, just use it as a chat box or help you refine your writing, just ask questions to JGPT. And other than this natural language, we also see the huge breakthrough of artificial intelligence in protein structure prediction. So we can use the AI to discover a new protein or even kind of like the structure analysis. And there are also a lot of applications trying to use AI for disease detection in order for early detection and early intervention in order to help patients to have a better, healthier life. And also the drug discovery is also growing pretty fast, like with AI, we can kind of like help a human as an assistant in order to more efficient and faster to find all these new drugs in order to prevent or cure different cancers, something like that. So the power of artificial intelligence is not just only being applied to different domains. And this year, the Nobel Prize in both physics and chemistry also confirms the contribution of the huge impact of AI in this fundamental research. So we see that the Nobel Prize is awarded to AI scientists in both physics and chemistry. And this is actually quite exciting for AI researchers, because it seems like there is a good collaboration, interdisciplinary collaboration between AI ways and those fundamental, principal research areas. And then now the question becomes, where and how can AI help diabetes care? So based on my 10 plus year AI research, we want to know how we can equip AI with all this clinical knowledge for glucose control. And based on my diabetes, like patient experience, we also want to infuse existing physiological knowledge into the AI design in order for it to better help the specific disease diabetes. So the first section that I want to deliver in today's talk is about automatic carbohydrate estimation. And actually, this also kind of like discussed in the last session, and the researcher is focusing on the image. So as we all know that right now for especially type 1 diabetes, but including type 2 diabetes, we all know that the diet is so important, because it can significantly influence our blood glucose control. And then for type 1 diabetes, there is one task that for every meal we need to do, which is carbohydrate estimation. So if we take one meal, then we need to estimate how many carbohydrates are included in this meal. And based on this information, we can roughly estimate how many units of insulin we need in order to make our blood glucose within the range. So now most of the patients, they just do the manual estimation, which is in general very challenging, because it's hard for patients to memorize all this carbohydrate and nutrition information for the meals. And it definitely requires domain knowledge. But in the future, what we want is that we want automatic carbohydrate prediction without any prior knowledge. We just expect a very minimum task from the patients. And the output, the prediction from the model, can not only focus on carbohydrate, it can also give you the information of glycemic index, which matters how fast the food will be absorbed in your body and change the blood glucose. So for example, even if you take the same amount of carbohydrate of sugar and rice, they are both 20 grams, then how much insulin that you should give yourself and when you should give yourself the insulin is supposed to be very different, because the absorption rate between rice and sugar are hugely different. OK, still, let's see, this is a standard task. This is one meal for my breakfast. And now I need to do the carbohydrate estimation, because I need to give myself insulin. So there is one thing, one way that I can do, that is I just Google search the carbohydrates in each food items, like the carbohydrates in the sliced butter toast, the carbohydrate in two eggs scrambled, the carbohydrates in around like 12 blueberries, and the carbohydrates in five strawberries. And then if we do this kind of thing, we need to add all these above carbohydrates together. And this can give you a rough estimation of your carbohydrate information of your meal, but it definitely increases a huge amount of time. And in reality, what I really do, I just escape all these processes and just take a random guess. So I'm kind of like overconfident, OK, so as a patient, I'm just so annoyed with those calculation, Google search stuff, and then to get the information that I want step by step and I just want to save time. So I just give a random guess and the input of the carbohydrate into my hybrid closed loop system and then start eating. And then afterwards, OK, that's a challenging part, because usually my body glucose wouldn't be that good. So I need to keep checking my glucose monitor every five minutes. I'm just trying to see whether I need to take further action. But unfortunately, you wouldn't be always available after the meals. Like as a professor, I still need to teach. I sometimes need to meet with my students. And I also have this kind of like a talk. And during this kind of meetings, it's hard for me to keep checking my blood glucose. And then let's see how we can do it with the power of AI. So given the same meal, like the breakfast, what are we trying to do is that we will ask the patients to describe using the speech or text input to describe what I'm taking for this meal. Like I'm eating two eggs scrambled with a slice of butter toast and a slide of five strawberries and around 12 blueberries for my breakfast. So this is kind of like a simple description. Whatever you eat, just describe it. And then the model will immediately give you the output of the food of the carbohydrates of each food item and also tell you the total carbohydrates in the meal. And in this in this example, that model's output is a large language model's output is like two grams for two grams of carbohydrates for the eggs, 13 grams for the butter toast, four grams for the strawberry, around two grams for the blueberries. And then it will also help you to do the simple math and it gives you the total carbohydrate. So if you just don't want to know the final number, then it's kind of like 21 grams of carbohydrates. But if you want to learn more about the carbohydrate in each food item, the model can also give you the model's reasoning path, how it derives to the final goal, like 21 grams of carbohydrates. So we see that this is actually doable in the existing LLMs. Then the next question becomes, OK, LLMs, if you ask this kind of question, it will always give you an answer. But how accurate are the carbohydrate estimations from LLMs? So it's not like if the model's output is still kind of like a similar as my random guess, then the power of LLMs are actually not truly a lot, right, to help us in our daily lives for this meal intake. So in order to answer this question, how accurate are the carbohydrate estimations from LLMs? What we are trying to do is that we develop a robust data set to include many different types of meals and then to evaluate how precise the model's predictions are. So here we build up in our very recent work, NutriBench, we build up the first benchmark for evaluating LLMs in carbohydrate estimation. So the NutriBench is built upon the real world global dietary intake data. But of course, this kind of data, they are not a natural language. They are just more like tabular data. So it's just like a food item for each one food item. It's kind of like for one meal, you might have multiple ingredients and food items together with its carbohydrate. And then our task is to just based on this information to generate the real world meal descriptions, natural language based meal descriptions. And in total, we have over like 11,000 meal descriptions, which covers from 11 countries. We like here, like American is one country, but other than American, we also have the countries from the Asian, from the Africa and South America. And for each meal, we also have the four micronutrition labels, including carbohydrates, the proteins, the fats in the food, because we know the fats actually also matters, and the calories. This kind of like different types of nutrition information can not only help people with diabetes, but also help people that want to live a healthy life by controlling their diet. So giving a full picture of the nutrition information of your food. And then we use, based on this data set, we use GPT-40 mini to generate the natural language based meal descriptions from the dietary information, the real world dietary information. And this information originally just in the tabular format. And then we convert it or transform it into the natural language. And we also perform human verification to ensure that the generated descriptions by the GPT-40 are accurate without like a hallucination or some misinformation. Here I present like two examples about the generated meal descriptions in NutriBench. So for example, for breakfast today, I have cheesy egg omelet made with oil, a pork sausage link, a slice of plain French toast, and a tablespoon of light pancake syrup. And then for another meal, like the lunch, I'm having a cup of bottled water, a piece of thin crust pepperoni pizza from school, and a cup of reduced sugar chocolate milk. So like the users can just interact with a model to describe what they have eaten for this meal, which is supposed to be complementary to the image domain. Because for the image domain, sometimes even if the model accurately identify the food items, the serving size, how large the food item is, doesn't matter a lot for carbohydrate information. And also for sometimes there are food covered by food. And then when you take a photo, it's hard to identify those covered food items. But by using the language as the information, we can just ask the patients or users to describe what they have eaten. And the input can be either by talking, the speech modality, or if the time is not good for you to speak, then you can just type it as text. And then we benchmarked 12 different state-of-the-art LLMs on NutriBench, just trying to have a picture of how good the model is right now. So these 12 models include the LAMA model, which is an open source model from Meta, the GEMMA model, which is also open source LLM from Google, and the QUI model, which is also one type of state-of-the-art LLMs. And other than open source models, we also evaluate a closed source model, means that you cannot get access to the model details, the parameters of your model. These include the GPT family, like we evaluate GPT-4-O, GPT-4-Mealy, and we also especially evaluate one medical domain-specific model, which is called OpenBioLLM. So this large-language model is actually pre-trained on a lot of medical information. This medical information can be very diverse, covering different diseases, not specific to diets and diabetes. But in general, we want to see if we train a model on those diverse medical knowledge, whether it can help one specific nutrition estimation task. And then, as we have shown before, if the user describes the meal description, like at lunch, I treated myself to a cup of bottled water, a piece of thin crust pepperoni pizza from the cafeteria, and a satisfying cup of chocolate milk that was low in sugar. So then the model can directly give you the answer about the total carbohydrate. In this example, it's roughly like 36. But this is just the basic method. You describe your meal, and then the model will just give you the answer about total carbohydrate. And then we also try something even more advanced, which we call as chain of thought. So in general, it will infuse the step-by-step reasoning capabilities of LLM in order to improve the accuracy and transparency of model's prediction. So when we use the chain of thought reasoning, then we can see that the model will not only just give you the final answer, what is the total carbohydrate in the meal, it will also give you the reasoning path. So it will first analyze this meal consists of one cup of bottled water, one piece of thin crust pepperoni pizza, and one cup of chocolate milk. Then it will disentangle this problem one by one by providing the carbohydrate for each food item. Like one cup of bottled water has zero grams of carbohydrates, one piece of thin crust pepperoni pizza has approximately 30 grams carbohydrates, and one cup of low sugar chocolate milk has about 24 grams. So the total carbohydrate in the meal is zero plus 30 plus 24 equals 54 grams of carbohydrates. So this kind of like a reasoning confers to verified by our later experiments that we will show first improve model's accuracy in making those carbohydrate estimation. And in the meanwhile, it also gain users more confidence because it's kind of like make the prediction more transparent. I'm not just give you one number telling you how much carbohydrate in the meal. I tell you how the food is consist of like different food items, and how the corresponding carbohydrate for each food item. This would be very beneficial for users if they truly want to learn those nutrition information for each meal. And in the meanwhile, if the user is just a super busy, you know, like I can also choose just to look at the end of the prediction. I know it's just a 54 grams. And the response speed of the models of these two methods are roughly the same. It's just like a within five seconds or 20 seconds, you can just get the answer. And not only the basic answer or the chain of thought, we also try evaluate another method, which we call it as retrieval augmented generation. So the general idea of REG is we retrieve food information from a food database to ground the predictions of LLMs. So as we know that we can actually get from the US database about all these food, the carbohydrate for all these food items. But of course, for all these food items, they are quite independent. There wouldn't appear like new information of new consistent of multiple different food items and corresponding with the carbohydrate. No, you don't have this kind of information. But if you want to get the information for each food item, then you can get a lot of like verified from the government database about the nutrition information. So here with a retrieval augmented generation, we try to use, take the advantage of this available database. So that is given a meal, we will first process how many food items, what are those food items consisted in this meal. And then for each food item, we will try to retrieve similar food items that you can get from the database and then get the ground truth from that database. So, for example, for this meal, we know we have the water. So then you can easily retrieve the water information, the carbohydrate information for the water, which is zero zero gram carbohydrates. And then we also have the the pepperoni pizza. So you can also retrieve from the database, which is a traditional thin crust pepperoni pizza, like 100 grams of this food contains 28.39 grams of carbohydrates. So one thing to note that in this kind of like a database, it will have some fixed serving size. It is either like the nutrition information will be either associated with one gram of food item or it will just be some natural serving like the chocolate milk drink. It will have like one cup, which roughly in the database is 248 grams of the food contains 26.49 grams of carbohydrates. So it's kind of like the general idea is that when you ask your model to give you your estimation of the nutrition information, you first allow your model to access the database and retrieve the corresponding information, providing them as a context for its prediction. So, of course, like it doesn't guarantee that for any food item you can find from the database, but for a lot of like common food items, like usually you can find this nutrition information from the database and then ask the model to process and integrate the information to give you a better grand data prediction. So this is a rack-based. And then with a retrieved context, we can still combine it with a traditional method and the chain of thought to ask the model give you a single answer or give you the reasoning path of the final total carbohydrate. So here we introduce 12 different LLMs. And for each LLM, we also try these different types of prompting techniques, like chain of thought and rack, in order to show how good it is to do the nutrition estimation. And here is actually the result. So in this figure, the x-axis is answer rate. So because when you prompt the model to give you the answer, the model sometimes can also give you no answer. So it's kind of like, OK, the information is not enough. So I don't know what is a carbohydrate in the meal. So in general, this kind of like no answer is annoying, because I already described a meal. I expect the model give me a precise prediction rather than keeps telling me, I don't know. I don't know, right? So here, the x-axis is kind of like trying to evaluate how good the model in estimating the given meal description, how good the model will respond to you with a carbohydrate estimation. And 100% means that 100 meal descriptions have been answered. Whenever you give a meal description, the model will always tend to give you an answer. The lower the answer rate means that there is more no answer from the model. And the y-axis is accuracy. So here, we set a threshold with 7.5 grams of carbohydrate. Whenever the prediction is within 7.5 grams of carbohydrate, we treat the model's prediction is accurate. The reason that we choose 7.5 grams as a threshold is that, on average, it corresponds to 0.5 unit for type 1 diabetes, which is supposed to be kind of like acceptable as long as the error is within this range. And then here, if we look at this figure, we can see that we actually evaluated many different models. And the best model that we achieved is on the right corner, right top corner, highlighted in the red circle here, which is GPT-4.0 model, the largest GPT model with chain of thought. So this is, and actually, if we look at the accuracy, it is around like over 65 accuracy with around 100% answer rate. This is actually quite exciting because it seems like the model, even though it is not trained on any nutrition information, this is just a general training model. But then it seems to perform pretty well in this specific nutrition estimation task. And even more excitingly is that we invite three human nutritionists, like human dietitians, to do the same task as the models. That is, we ask the dietitians to give a carbohydrate estimation for these meals. But of course, it's hard to ask the dietitians to evaluate the total over 11,000 meal descriptions. That would take too long time. We ask them to evaluate a randomly selected 100 meals and ask them to evaluate the carbohydrate information. And then we realize that the average, the three human dietitians, they achieve around like a 46 accuracy, which is still much lower than the GPT, like the GPT-4.0, the best model that we have. And it is actually, there are many other models, like other LLMs, like the GEMMA or LEMMA models, when we use the chain of thought techniques or RAC techniques, it is actually outperform human dietitians. So this result, whenever we start comparing the models with the human experts, we start realizing even more that the model is actually really good at doing this job because the model can actually memorize all this detailed information and then just the process integrate the information to give you a more correct nutrition estimation. And other than this kind of like a numerical evaluation, like the answer rate, the accuracy, we actually also do real-world risk assessment. So like the general idea is that the nutrition estimation is still the intermediate stage. We want to see if the model is doing a good job or a bad job in nutrition estimation, how it will affect the patient's blood glucose. So then we use the TIPOR data science simulator to simulate how a virtual patient's blood glucose will change over a period of time in response to the external events like the carbohydrate meal intake. So here is a plot for the best model that we have achieved, the GPT-40 denoted in the blue color, and the three other plots, which is the glucose plots, if you use the estimation, the carbohydrate estimation from each nutritionist. And then the x-axis in this figure is a time, which is around like six hours after you take the meal, and the y-axis is a blood glucose level. So if the blood glucose fall into the middle range, the green range, that means like the blood glucose is still in the range, in the normal range that we would expect, we would hope our blood glucose can be. And if the blood glucose trace falls into the red zone, that means the patient, the virtual patient will suffer from hypoglycemia. And then if the blood glucose goes beyond into the yellow zone, which is hyperglycemia, like the high blood glucose sugar, like in general, like if, for example, the type 2 diabetes, if they keep, like not to keep their blood glucose under control and keep suffering from hyperglycemia, high blood glucose, there will also be associated complications. And then we run the simulators for 20 virtual patients, and we have different scenarios, like their initial blood glucose, their different carbohydrate intake ratio, insulin ratio, and like their, like what's the time they take the meal, all this different information. So in total, we simulate 44,800 simulations. So then we evaluate the performance under two major scenarios. The first one is that the patient is not using a pump. So then that is, the patient needs to calculate the insulin dosage manually based on the estimated carbohydrate. And based on their carbohydrate insulin ratio, which is a simple, like X divided by the carbohydrate insulin ratio, they will give themselves a certain amount of insulin dosage. So the main thing is that if this kind of patient rely on the carbohydrate to give themselves insulin, then they don't have a pump to adjust their blood glucose immediately. So it's kind of like, whatever the action that you have done, if like during, like for using the insulin pen to give yourself insulin, you cannot do further things, like relying on insulin to help you control your blood glucose. And the second scenario is actually the patients, we're targeting the patients using a pump. So that is, even if you are using the insulin pump and also with the AID system, then you still need to give the AID system an estimation of your carbohydrate. And then the controller in the AID system will give yourself like the insulin and the controller can keep adjusting your blood glucose based on like your current blood glucose. So we evaluate these two scenarios, like covered in this 44,800 simulations. And we here, we report the time in range, the percentage of time in range, the percentage of time below range, and the percentage of time above range. And the last one is just like a blood glucose risk, which can be like a low blood glucose risk and high blood glucose risk. So for the TIR, TBR, TAR, we always wanted the model to have a better performance, but here we specifically evaluate our best model, GPD 4.0, together with Jennifer and the three nutritionists. So we can see that actually GPD 4.0, the model, the LLM, actually achieves the highest TIR time in range and the lowest TBR, which is quite good because we actually want to prevent a TBR a lot because it's so dangerous, like patients will feel very uncomfortable, like when they have low blood glucose sugar, and they also need to take further like a sugar or snacks in order to compensate this low blood glucose sugar. And then the last TAR is actually when nutritionist is doing the best job for time above range, but compared to 24 and the 25 that achieved by the model, it would say that actually the model is still doing a decent job in controlling your blood glucose within the range rather than going high, going above the high range zone. And then if we look at the blood glucose risk, so you can see that combining all these TIR, TBR, TAR, the model achieved the lowest risk compared to the human experts. So this is actually quite exciting because even for those virtual patients-based simulations, we can see that the model LLM is actually doing quite good job in a nutrition carbohydrate estimation in order to help people to control their blood glucose. And then after this kind of like all this evaluation and benchmarking, we have a sense that okay, it's so cool that LLM seems to be a great potential for help patients with diabetes in control their meal intake by estimating the carbohydrate. And then the next question is that, can we just fine tune the existing LLMs to become LLM nutritionists? That is, we want to fine tune these existing LLMs with the general natural language knowledge on the specific nutrition information datasets. And then in order for the model to have more, develop more domain knowledge on the food items and then the nutrition information. So here in this table, we evaluate the existing LLM and the fine-tuned LLM. And we try to compare the performance in terms of the mean absolute error. The lower means the better. And the higher accuracy means the more precise prediction. And the higher answer rate means the model tends not to give you no answer, like when you describe the meal description. And then you can see that compared with the original LLMs and our fine-tuned LLMs, we definitely significantly improve like the MAE with a lower MAE, higher accuracy, and a higher answer rate. So this is also quite exciting. That means we can actually build upon the existing LLMs to develop our own LLM nutritionists in order for it to better help patients control their blood glucose. Okay, and this series of work about NutriBench is actually the paper is available online. We also have our website available. And then we also release the NutriBench, the data with the nutrition information. And this is actually like collaborated with, like done by my amazing PhD students and my collaborators, right? So because of them, like we have this amazing work now showing to us. Okay. So after all this discussion about the research that we have done on the meals, the second most significant event about diabetes is exercise. So we know that actually when we do different types of exercise, unfortunately, we're suffering from hypoglycemia more significantly. Because in general, if you do jogging, biking, or even walking, like your blood glucose have a higher chance to drop immediately. And then unfortunately, you will just go under like the normal range, your blood glucose. So then what do patients, including I, do now if I exercise? So again, random guess. So I kind of like know that, okay, I'm suffering from hypoglycemia when I start running. So then I know that I supposed to reduce my insulin or the basal insulin to some extent before my exercise. But the question is that I don't know how much insulin I should reduce in order to keep my blood glucose within the range, rather than either hypoglycemia or because I reduce too much insulin, and then I unfortunately also suffer from hypoglycemia. So then to me right now, because I have no knowledge yet, like I believe most of the patients are like me. So then we reduce our basal to a random degree. So then I start running. And then unfortunately, this random degree usually is not enough. And then I start running into low blood glucose. And then like I have to stop from the treadmill and start eating those sugars and then just keep blindly eating. So in general, eating can give me a lot of joy. But when this carbohydrate is rescuing meals, like it just give you tons of stress, no happiness. You just like you physically feel uncomfortable and you just keep eating things trying to compensate the drop of blood glucose. And usually whenever you have hypoglycemia and you start eating, you will unfortunately also run into hypoglycemia. At least this is my scenario. Because when I blindly taking extra sugars, chocolates, whatever, and then after like half an hour or one hour, my blood glucose start increasing insanely. So then I will run into hypoglycemia again. So this is kind of like a back and forth. It's just so annoying and very stressful for all the patients. So therefore, in order to understand more about the glycemic effect of real world exercise, now we're working on a project which is to investigate how exercise can affect our blood glucose. So we mainly focus on the T1-DEX-C data, which includes a lot of unstructured real world physical activity sessions. It has over 450 patients, like they do different types of exercises. And here we pick the four most frequent activity types. They're walking, biking, jogging, and strength training. And for each, here we also display how many sessions that we have for each activity. And then we check how blood glucose change after exercise and before exercise. So based on this table, you can see that exercise generally needs to drop in blood glucose. For all types of exercise, they will decrease our blood glucose to different degrees. And correspondingly, because the exercise will lead to blood glucose drop, it also increase the TBR and hypoglycemia risk based on this T1-DEX-C data. That is for each activity, how many patients are suffering from TBR. And then we have two ADA abstracts, which is mainly to investigate the different factors which can affect the glycemic effects of real world exercise. So for the TBR or hypoglycemia risk, we investigate the factors including the initial blood glucose, the insulin on board. Do you take the meal before your exercise within two hours? How long do you take the exercise? This is a duration. And whether you take the extra snack before or during your exercise. We also have some population level factors like your sex, female, male, your age, and your HbA1c. So we want to see these different types of exercise, the different types of factors, how they can affect the potential of you getting hypoglycemia. And actually, we see our major takeaway is that first, the glycemic effect of exercise varies with different types of activities. As shown in the previous table, we see that if you do different types of activities, like the running and strength training, your blood glucose will just drop to a different degree. And longer duration, lower initial blood glucose, and higher insulin on board have larger glucose reductions and higher hypoglycemic risk. So that is, if you do your exercise for longer time, like one hour or one hour and a half, and if you start with a relatively low initial blood glucose, like maybe around 80, or even like 100 compared to 150, and if you also have more insulin on board when you start the exercise, then you will just suffer more for these hypoglycemic risks. And we also observed that sessions with the carbohydrate intake had a lower initial blood glucose and correlated with higher hypoglycemic risks. So it's kind of like if you start with a lower initial blood glucose, and you also have some carbohydrate, which also suggests that there will be more insulin on board, then you will also have a higher chance of hypoglycemia. And then for this ongoing project, like all this data analysis gave us tons of knowledge about how exercise affects our blood glucose during and after exercise. And then we are paired with Tai Po to work on how we can design a pre-configured activity-based presets in order to compensate the blood glucose. So the existing Tai Po loop system, it has one action, which is to raise the target. Originally, for example, the target is around 110. So 90 minutes before your exercise, you can start raising your target to be 150 in order for the controller to deliver smaller insulin into your body. So this is an existing loop system to compensate for the effect of exercise. And for our ongoing project, other than raising our target, we also propose a pre-configured preset, which is kind of like whenever you start raising your target, you also start reduce your basal insulin to a certain degree in order to reduce the insulin in your body to compensate as a drop of blood glucose. So this is actually a three-year project funded by Helmsley Trust. Great thanks to their support. And this is also cross-institution collaboration with Tai Po, with Stanford, with University of Trenton. And for this three-year project, in the first and second phase, we are focusing on design the static activities specific preset based on our analysis on T1-dexed data. That is, we will design one insulin preset for each most frequent activity type. So then after the static preset design, we will also move to the dynamic preset design that will integrate all these important factors we have analyzed into the AI model to better control their blood glucose. So that is, if you exercise for different time, like different durations, and if you start with a lower blood glucose or higher blood glucose, you will have the corresponding preset recommendation based on your current stage when you start exercising. But of course, this is kind of like ongoing in the later phase. We just started this project last November. Right now, it's kind of like after one year. And then, together with the algorithm, we also have the clinical study to verify the effectiveness of the static preset design integrated into the type 4 loop system. So this, we're also together with the education plan to the patients, just educating them how to better control their blood glucose by turning on the preset. Okay, so for today's talk, I will mainly discuss how we design the static preset. So the general idea is that we want to design the static preset P to compensate the effect of exercise. So assume that based on the exercise guidelines, clinical trials, and the real-world usage, we will ask the user to turn on the preset 60 minutes before the activity and then turn it off at the end of the activity. So the whole preset duration will be 60 minutes before activity and the end of your activity. And then the effect of applying the preset, because we already see that the insulin, the blood glucose will drop, so the preset is actually within 0 to 1. So if P equals 100, it means standard delivery. If P equals 0, it means that you suspend your insulin pump. And if P equals 60%, it means 60% of standard delivery will be delivered, like before and during your exercise. And so the effect of applying P will be twofold. The first one is that it will reduce the delivered insulin from the original I to be 1 minus P times the insulin. So it's kind of like reduce your insulin delivered. And the second one is that it will also correspondingly change your insulin sensitivity factor, which we know that based on some physiological knowledge, we know that when you exercise, the insulin sensitivity factor is actually increased. So here we can see that the insulin sensitivity factor will also be scaled by the preset. And then based on these two effects, we can estimate the blood glucose at any time T after applying the preset with this equation. So that is based on the original glucose data that you can get from T1-DEXC. You will also take into the scaled insulin and the scaled insulin sensitivity factor how these two components will affect your blood glucose in the later stage, during and after your exercise. And we obtained optimal preset for each selected sessions for all four activities from T1-DEXC data. And we use a medium of the optimized preset for each activity. And here we list the preset designed for each type of activity. So we can see that actually, like for biking and jogging, the preset is actually very similar. But in general, this anaerobic exercise will need a smaller preset compared to the strength training, which is anaerobic exercise. And we also systematically, comprehensively evaluate when we use the designed preset, how it can help reduce hypoglycemia risk. Here we compare without preset, which is just from the general data. And with the preset, that is, if we use a preset, how the blood glucose will evolve. And here we use three metrics, which is TBR, that is glucose is within 54 to 70. And TBR 50, that is a glucose will drop below 50. And the hypoglycemia risk, which is there are three consecutive glucose readings is smaller than 70. Because we know that every five minutes, you can get one glucose reading. So if three of them are smaller than 70, in the duration of the preset, we will say that this session actually has hypoglycemia. So the without preset, we denote them in the blue bar in the figure. And then the red bar represent with preset. And then you can see that for the hypoglycemia risk, by applying the preset, we significantly reduce the three metrics actually for different types of exercises. And now we're also working on verifying the effectiveness of the static preset design in the simulated environment using the typo loop controller. So we want to see that if we turn on the preset together with the controller's reaction to the blood glucose change, whether it can also achieve the best performance. And so far, the preliminary result seems to be quite promising. But of course, for our daily life, we're not only take meals, we're not only just to do exercise, there are many different type events. So we might get sick, we might just travel to different time zones, and we might be stressful sometimes because of our work or human relationship. And for women, we also have the pregnancy and periods. This kind of a different time or different life events. As patients, we know that they just change our blood glucose in different degrees. And the final goal is that we hope to use design AI models to integrate, to be smart enough to get familiar with our daily life patterns, and then just to become a close friend with us. But in the meanwhile, by knowing all that happens, all the things happening in our life can give us personalized recommendations. Yes. So then the next thing that I want to briefly talk about is about a personalized assistancy, which is definitely what AI can help us do. So right now, so for the patient, what we have is we will go to the doctors, and the clinicians will help us to manually define the patient profiles with their domain knowledge. And these patient profiles can be, how much is your insulin sensitivity factor? How much is your carbohydrate insulin ratio? How good it would be to set your basal insulin to steroids? All these patient profiles are manually set up by the clinicians. But we all know that these kind of patient profiles are not that precise, because even for clinicians, they just can derive this knowledge based on average people, most of their patients, rather than especially for you. And then for AI models, what do we want to use it for? We want to use AI models to learn from our historical time series data, our daily events data, our insulin data, and can learn how to customize the patient profiles from the data without any prior knowledge. This is definitely doable, and it would also be very tailored to the personal needs, and then to help you give the insulin recommendation more personalized and tailored to your needs in order to help you control your blood glucose. This is one thing that we can start working on, and personally, I'm also very excited we're working on. And then the second, the last, but definitely it's just the last topic that I want to cover about the potential of AI models in diabetes control. This is the continual improvement. So now for the existing AID systems, whenever you set up these patient profiles, usually you don't update it, even only if the AID system is really doing a bad job, and then you have to go to your clinicians to do some correction based on those kind of messed up controlling. But usually, after everything is stabilized, after months of this stabilization, and then you will never change anything. You just keep using the same patient profile, same factors, the same parameters, and then ask the AID. And of course, it's not perfect. It's just kind of like there's still extra burden, a lot of stress, and we can consistently see the blood glucose fluctuation. But in the future, if we ask AI to be an assistant for our blood glucose control, it is actually, as long as the AI assistant stay with you longer, that means it can get more data from your past days. So it's kind of like collect more historic data. And with the more historic data, we can always easily fine-tune the model to be better tailored to your current state and help your blood glucose control. So that is, it is not like one time turn on all these parameters and everything is just fixed there. But instead, as long as you carry your AI models, AI assistant for your diabetes care, then you can keep, you can see that there is a keep improving, they can keep improving your glucose control because they know more data, get more data from all these wearable sensors and devices, and then they can become even smarter. It's just kind of like the friend gets to know you better and better. And of course, know how to give you suggestions tailored to your personal needs. So, so far for these two components, we haven't, like our lab haven't started yet. There are some ongoing projects, but based on the time limit, I don't have, I don't want to talk too much more, but here is a summarization of where machine learning or AI can help diabetes care. So I specifically focus on the two major types of events, which is meal, about automatic carbohydrate estimation using large language models, and exercise how to design the preset for the existing AID system in order to better control hypoglycemia. And like in the long-term run, we also wanted the AI to become our personalized assistant and keep improving based on more and more historical data, and becomes a closer and a better assistant for our diabetes care. And then thank you very much for your attention. And we'd love to hear about your questions. We'd love to seek collaborations. And also if you are patients or have some patient's data, we'd love to discuss how we can make the better use of the data for AI design in order to help more patients. So I guess the coordinator, we are just coming in for the question answering sessions. Yeah. Thank you for the engaging talk. This is an important application, the most challenging aspect of type 1 care, and certainly provides beacons of hope for many person living with type 1 diabetes. And may I start off with my question? For the neutral bench data, they are from different countries. And there are many different terminologies used for the same type of food. And the dishes with the same ingredients may use different names. How do you address all this issue? Oh, yeah. So actually, the neutral bench, we developed from the WHO data set. And there, it is still just to support the English for that data set to describe the meals from different countries. And actually, we're also expanding this neutral bench. In this talk, in this existing work, it covers around 11. But now we actually have over 20 different countries. They are still support English. And then regarding your question, how to align them like using different language, I assume, like people from, is that? The terminologies, the terminologies, like some food, like tea. Tea in UK means a different thing, a cup of tea. And tea in maybe Australia, they mean a dinner or something like that. So the same word may have different terms. So how do you align all these different terminologies? OK, this is a very good question. I think like a base, the current neutral bench doesn't design specifically to address this problem because it's kind of like the data that we get from the WHO. In general, it is still just reported by local people from each country. You know, like they report the food meals that they take. But I believe there are some subtle things about the terminology of different food. I feel like if we scale it even up, like for each country, then this problem will become much more significant. I think we should definitely consider it when we try to fine tune our model, because right now when we fine tune our model, we still mainly focus on the US data because it's very large scale. But for the neutral bench, it is only for meaningful evaluation. So it's not that large yet. Yeah, but very good point. For me, I feel like I don't have a clear answer to you how we can address it, because if it's like the model is relying on data, then the best thing that we can do is just collect all this data and enable the model to identify like the different terminologies of food items from different countries. Yeah. The other thing is, when you build your model, are you using like terminologies based on concept rather than true definition, like from the medical community or something along similar lines? What do you mean by concept? Okay, when we do the PubMed search, we are looking at, searching through the concept, we use a certain word like diabetes. Right. And when it comes out, diabetes insipidus may come out, type 2 diabetes may come out, and other related terms with diabetes based on the concept of diabetes. The whole list will, well, we may see about 30,000 hits that comes out through the PubMed. How do we like focus on the actual thing, the actual terms that we want? Because in the NutriBench data, there are a lot of data entry basically, and sometimes it may be quite confusing if we are trying to use that. Yeah, so for the NutriBench, I feel like it's quite general, because like originally, like I kind of like encourage my lab to work on this area is for type 1 diabetes, because we know that we have to do cup counting every day, right? But then after we started this line of research, we realized that it is not limited to type 1. Like for type 2, they also need to be very careful with their diet. And for people with obesity, then they also need to know, like have a better knowledge of the food information. So then like even for some like cancer patients, like whenever the doctors would suggest you pay attention to your diet, then this kind of like nutrition information in diet, in meals, is just very helpful. So then for me, I feel like this, especially for NutriBench, is supposed to be a very general, like under, if you truly want to assign a keyword of it, it's supposed to be under nutrition, rather than any fixed type of diabetes. Yeah. Thank you for the enlightening. And okay, the other question that comes out in my mind is, you have developed different LLM models in different aspects of diabetes care. And I think the next step would be, are you planning any LLM model learning with real person living with type 1 diabetes? That would, I mean, your model would be more accurate targeting at those patients? Yeah, yeah, yeah. So definitely. And I think that's also one kind of like a benefit of being a patient. I think I will be the first one to try our model and then see how it can, because I have, I wear like the insulin pump and the glucose, the CGM. So kind of like I have all these like a sensor data, as well as like if I want to interact with the model more with my daily life, I can just talk to it. So yes, like that's definitely on our like, you know, like the long term, our like lab or my research goal. Definitely. Yeah. And I think that would also kind of like make all this model design to be deployed in real world, you know, like truly help patients rather than just a simple, rather than just focusing on epidemic research, but instead it is truly help patients. So that's also why I kind of like a really seeking collaboration from clinicians because of my domain is focusing on AI. I think I'm very like my lab and we are very good at designing new models based on our AI experts. But we are not clinicians, you know, like we don't directly talk to patients. I do know a lot, a lot of patients actually, because I'm a patient, but I feel like we definitely want to view the collaborations with clinicians and in order to provide a complimentary perspective, perspective, because on our exercise project, that is a kind of like interdisciplinary collaboration with Stanford clinicians. And they are so helpful. I'm very grateful for this collaboration. They kind of like gave, provide a lot of like insights from the exercise modality. And then we kind of like integrate those insights into our model design. Yeah. And the other thing is the devices like this, if you apply to real life patient, we are considering this as medical devices. And what are the regulatory standards for this devices before they can be tested in real life patient? Yeah. Yeah. OK. So this is also a very good question. I think like for that part, we need, we need to say collaborations with industry actually, because we don't want to design a new AID system or this life event integration or this kind of like a nutrition information supposed to be complimentary to the existing AID system. So then it's kind of like enable one more feature in the existing AID system powered by AI and in order for it to better control. But I believe that if we build upon the existing AID system, the FDA cleared all this process might be relatively shorter compared to you build everything from scratch. But in general, for me, I feel like, yep, AI can definitely revolutionize the diabetes scale. But all this like physiological knowledge developed from all this clinical study in the past decades and all this kind of like hardware design or this sensor design and the existing AID system is definitely a huge breakthrough and help patients a lot. And at the current stage, because of huge success of AI, we just want to see how we can make the improvement of diabetes care to be even better and better. You know, it's kind of like we know the patients are already kind of like a much easier life compared to 20 or 30 years ago, you know. But we just want the patients to live a fully, freely life as normal people. I think that would be our final goal. Yeah. Great work. Thank you. Yeah. Any more questions from the floor, please? Okay, looks like the lecture is very good, it's very clear, it gives us a very good view of the challenges that is facing a person with diabetes and also the hope that AI can do a lot of things for this group of patients. Yeah, so definitely, I'm not sure, but I believe the audience from this talk are coming from different backgrounds, so in general, we're very open to collaborations, either your patients or clinicians or industry collaborators, we just love to start checking, and if you can also have some data that you feel like it's supposed to be able to help AI design, then just super appreciate that you contact us. Yeah, okay, thank you very much for the invitation, yeah. Xiaoping, there are a few questions in the chat, I'm not sure if you were able to ask these already. Okay, actually I don't see anything on the chat. Oh, okay, I can step in if you'd like, so we have one question here, any suggestions on the use of AI in obviating hyperglycemia following exercise? Sorry, can you say the question again? Yes, any suggestions on the use of AI in obviating hyperglycemia following exercise? Oh, yeah, so right now it's kind of like, because when we design the preset, right, it's based on our clinician collaborators, we decide to turn it on one hour before, and like turn it off at the end of exercise, because personally I also see that if this kind of like reduce of insulin is not that good, then you're not only, either you will suffer from hypoglycemia during exercise, or you might also suffer from hyperglycemia after exercise. So that's kind of like the whole project is trying to make the preset design better and better. But for our evaluation, actually, we not only evaluate the glucose during the exercise, we also evaluate one hour, or even like a two hour, four hours after exercise to just trace the blood glucose change. Yeah, and when, I think right now we're focusing on the static of preset design for each type of exercise, and the general milestone is that starting from early next year, then we will start our clinician start a clinical study. And in the meanwhile, we will also move to dynamic static design, which is also more AI powered at that time. I believe I can give you a better suggestion when we start moving to the dynamic one, because they are, it's just in general very complicated problem, right? Like your initial blood glucose, how long you exercise, how many insulin on board, all these questions are entangled together, like these factors are entangled together. And the dynamic one is hopefully, hopefully, ideally, is kind of like based on your current situation, give you the personalized preset recommendation. Yeah. Okay, and I think Usha here has two questions for you. For exercise, do you consider VO2max as a factor affecting blood glucose? That's the first question. And the second, all right, okay, please answer. No, this is a clarification question. What is a VO2max? Can you, I think this might be one terminology. Okay, I think the, it's a terminology used in sports, in exercise, VO2max, I think the exercise capacity as a factor affecting the blood glucose. Right now, we actually mainly capture heart rate. So our metabolism model is also centered around the heart rate change during exercise, which is kind of like to represent how intense your exercise is. And I don't think that for the current stage, we don't consider, we haven't considered this factor into our model design. And I believe, I'm not sure whether there are some more preliminary result on this factor and how it affects the blood glucose, because this factor is not included in the data set that we are working on, the T1-dexy data. So that's why now it's not in the scope. Yeah. The second question here is for carbohydrate counting, do you consider counting protein due to gluconeogenesis, approximately 50% of protein will convert to glucose? For example, counting a big piece of steak without carb, blood glucose still go up. Okay. So this is a really great question. So I think for the meal one, it's definitely, we just started early this year, actually. So like a carbohydrate estimation is just the one nutrition. And we know that for the meal, it's more than carbohydrate and all this fat protein, they just kind of like influence your glucose in different ways. But that's also kind of a suggest how simplified right now, when we do all this carbohydrate estimation, if you ask a patient, because it's just a one cup, right? And also, as I said, in the future, we also want to incorporate a glycemic index prediction because we know that is also quite important for when your food will be converted into the blood, transforming the blood glucose. So then this is a super important question. Actually, we would definitely hope to incorporate the factors of different nutrition elements in the future design. And I think that part will also be paired with dietitians or nutritionists, because in general, it's kind of like we need those kind of domain knowledge in order to help us tailor our AI design to better help patients. Yeah. And Mia here likes to know, any of the things that you have discussed is available for use now? Yeah. So like the exercise, as I said, we are like as a clinical study, we started early next year, but I think that still need to be verified. And the nutrition estimation is actually, we started the research project early this year, and now the result is very promising. We are kind of like in the mode of like trying to find collaborations with clinicians to help us just recruit a small group of patients to get started on verifying how it help the patients in the real world. Yeah. So it's kind of like it's also on the agenda, but if you are either patient or like the clinicians, feel free to reach out to us for more detailed discussion on this part. Yeah. And finally here, Alina wants to know, do you have any assistant company started working on the incorporating of the concept that you described? No, like I think that like, because for me, I'm very AI perspective, and it's kind of like we finished, we kind of like the information that I shared in this talk are quite latest. It's just kind of like some of the, maybe just the last month we finished this component, and I kind of like introduced the state of art as a result in our lab, in our research. So we haven't like communicated with any insulin pump companies yet about this integration. But as I said, it's kind of like all these designs are quite complimentary to the existing AID system. So we definitely looking for collaborations. Yeah. All right. Okay. I think that's, all right. Thank you. I think, well, thank you for the great talk and the question and answer session. Yeah. Thank you for the invitation. Yeah. Feel free to reach out.

Data-Driven Machine Learning

Closed-Loop Diabetes Care

AI in Healthcare

Blood Glucose Management

NutriBench Dataset

Nutrition Estimation

Automated Insulin Delivery

Exercise and Blood Glucose

Personalized Diabetes Care

AI and Type 1 Diabetes

Insulin Presets