1 00:00:00,000 --> 00:00:05,433 One of the most, common applications of, Bayesian networks, or rather, one of 2 00:00:05,433 --> 00:00:10,539 the earliest ones that are still very much in use today, is for the purpose of 3 00:00:10,539 --> 00:00:15,776 diagnosis. And by diagnosis, I mean both medical as well as fault diagnosis. 4 00:00:15,776 --> 00:00:21,210 Now, this dates back into the early'90s, in, the PHD thesis of Heckermann, et al., won the 5 00:00:21,210 --> 00:00:26,316 ACM dissertation award. And a system called Pathfinder, which looked at, a 6 00:00:26,316 --> 00:00:31,880 range of different pieces of evidence in order to help a doctor diagnose, a set of 7 00:00:31,930 --> 00:00:37,260 diseases. And specifically, it was focused initially on lymph node pathologies, so 60 8 00:00:37,260 --> 00:00:42,576 different diseases, all sorts of different symptoms, and they tried out a bunch of 9 00:00:42,576 --> 00:00:47,957 different rules, methods for solving these problems. So the first one they actually 10 00:00:47,957 --> 00:00:53,338 tried, this way back in the early days of artificial intelligence, and they tried a 11 00:00:53,338 --> 00:00:58,587 rule based system. And, it didn't work really well. The second version of 12 00:00:58,587 --> 00:01:04,247 Pathfinder used the naive Bayes model, which assumes that all of the symptoms are 13 00:01:04,247 --> 00:01:09,837 independent given the disease. And even that really simple model got superior 14 00:01:09,837 --> 00:01:16,460 performance to the rule based system that they initially tried. Pathfinder three 15 00:01:16,460 --> 00:01:21,391 still used Naïve Bayes but it used Naïve Bayes with better knowledge engineering. 16 00:01:21,391 --> 00:01:26,633 That is they actually, they actually understood some of the issues behind what 17 00:01:26,633 --> 00:01:31,126 makes a system like this work well and what. Made and they've fixed it. So 18 00:01:31,126 --> 00:01:35,883 specifically one of the things that turns out to be really fundamental for the 19 00:01:35,883 --> 00:01:41,002 performance of any probabilistic modeling system is not to put in zero probabilities 20 00:01:41,002 --> 00:01:45,519 ever except for things that are definitions because once you put in a zero 21 00:01:45,519 --> 00:01:50,095 no matter how much evidence on the contrary you have you will never ever be 22 00:01:50,095 --> 00:01:54,853 able to get rid of it. Because anything times zero is still zero. And so here in 23 00:01:54,853 --> 00:01:59,549 the initial pathfinder tool they put in some incorrect zero probabilities for 24 00:01:59,549 --> 00:02:04,185 things that were very unlikely but not impossible and it turns out that, that 25 00:02:04,185 --> 00:02:08,775 gave rise about ten percent of incorrect diagnosis in the system. And also 26 00:02:08,775 --> 00:02:12,742 did better calibration of conditional probability which turns out to be 27 00:02:12,742 --> 00:02:17,259 important for knowledge engineering of the Bayesian network. So for example it turns 28 00:02:17,259 --> 00:02:21,887 out that it's a lot easier to compare for a physician, to compare the probability of 29 00:02:21,887 --> 00:02:26,515 a finding, a piece of evidence between two diseases as opposed to the probability of 30 00:02:26,515 --> 00:02:30,757 two different findings within a single disease. It's much, it's much easier to 31 00:02:30,757 --> 00:02:35,219 say "oh this is much more likely in this context than in that context" and it turns 32 00:02:35,219 --> 00:02:39,627 out that when they ask the physician to calibrate this way, they got much better 33 00:02:39,627 --> 00:02:43,847 estimate of the probabilities. Remind you this was way before they had learning so 34 00:02:43,847 --> 00:02:51,068 it was all hands constructed. And then, finally Pathfinder four was the full 35 00:02:51,068 --> 00:02:56,417 Bayesian network in all of its full glory. It no longer made incorrect 36 00:02:56,417 --> 00:03:01,574 assumptions about independencies between different, say, symptoms. I'll give them a 37 00:03:01,574 --> 00:03:06,731 disease. And that gave us an, both allowed them to make the model more correct and 38 00:03:06,731 --> 00:03:11,761 also it turns out as an unexpected side effect by allowing, say, a symptom 39 00:03:11,761 --> 00:03:17,046 variable to have more parents than just the single disease variable it actually 40 00:03:17,046 --> 00:03:22,140 gave rise to considerably more accurate estimation of the probabilities because 41 00:03:22,140 --> 00:03:26,725 the doctor could kind of think about different cases and didn't have to average 42 00:03:26,725 --> 00:03:33,021 them all out in his head. And this is one of the, I think, really compelling aspects 43 00:03:33,021 --> 00:03:39,257 of Bayesian network models which is that the Bayesian network model actually turned out 44 00:03:39,257 --> 00:03:45,121 to agree with the experts in an expert panel of physicians in 50 out of the 53 45 00:03:45,121 --> 00:03:50,689 cases. And these were hard cases. These were ones that you really needed the 46 00:03:50,689 --> 00:03:56,554 expert opinions on. It wasn't one that just an average doctor could necessarily 47 00:03:56,554 --> 00:04:01,973 diagnose correctly. And this is as compared to 47 out of 53 for the naive 48 00:04:01,973 --> 00:04:07,184 Bayes model. Also and significantly less than that for the rule-based system. Mind 49 00:04:07,184 --> 00:04:11,738 you, and this is an interesting and important. Aspect is that the Bayesian 50 00:04:11,738 --> 00:04:16,554 network actually outperformed the physicians who designed the model. And I 51 00:04:16,554 --> 00:04:21,568 mean it didn't outperform the expert panel but it outperformed the physician who designed it 52 00:04:21,618 --> 00:04:26,770 Because it was better at putting together all these different numbers in a way that a 53 00:04:26,770 --> 00:04:31,534 doctor just can't fit all of these different findings into his or her brain 54 00:04:31,534 --> 00:04:40,971 at the same time. So we talked about, the CPCS network. It's, one of my, favorite 55 00:04:40,971 --> 00:04:47,494 networks, because it's kinda big and hairy, and sorta kinda scary to look at. But 56 00:04:47,494 --> 00:04:54,352 anyway, the actual number of variables in this network is about 500. And each of 57 00:04:54,352 --> 00:04:59,794 them has, on average, about four values. So the total number of parameters, if you 58 00:04:59,794 --> 00:05:04,103 were to specify a full joint distribution, is about 400-500, so that's about, it's 59 00:05:04,103 --> 00:05:08,688 about four to the 500, or two to the power of 1000, which is more than the number of 60 00:05:08,688 --> 00:05:13,162 atoms in the universe. So obviously, one couldn't specify this as a complete 61 00:05:13,162 --> 00:05:17,582 joint distribution. Not to mention that the probability of each and every one of 62 00:05:17,582 --> 00:05:22,061 these is about, is as close to zero as makes no difference. Because it's the probability 63 00:05:22,061 --> 00:05:25,757 of, you know, 500 different, I mean events involving 500 64 00:05:25,757 --> 00:05:32,750 variables. If you were to if you were to actually construct a CPD for each of these 65 00:05:33,005 --> 00:05:39,700 for each of these variables the number of parameters would be about 133 million. 66 00:05:40,600 --> 00:05:47,441 Which is fairly better than two to the 1000 still much too large. And so it turns 67 00:05:47,441 --> 00:05:54,536 out that they made additional simplifying assumptions that we'll talk later on that 68 00:05:54,536 --> 00:06:01,631 allowed them to avoid a complete table representation of the CPD's and rather do 69 00:06:01,631 --> 00:06:08,902 a more compact one and that gave rise to about a thousand parameters. Which is 70 00:06:08,902 --> 00:06:16,495 still a lot but not but actually is tractable to deal with. So we already 71 00:06:16,495 --> 00:06:21,839 talked about the fact that these medical diagnosis systems have emerged from 72 00:06:21,839 --> 00:06:27,183 research and Microsoft built a medical diagnosis system. Various other people 73 00:06:27,183 --> 00:06:31,892 have built one as well. This has been a little bit slow on the uptake in the 74 00:06:31,892 --> 00:06:36,501 medical field because it doesn't fit naturally into a physicians, pipeline. 75 00:06:36,681 --> 00:06:41,469 Maybe now, with the advent of medical, of electronic health records, there will be 76 00:06:41,469 --> 00:06:46,258 more data entered into the computer so, these systems will be more common use. 77 00:06:46,258 --> 00:06:51,286 But, until very recently most doctors just wrote stuff down on paper and so there, it 78 00:06:51,286 --> 00:06:55,655 was very difficult to put this into the standard production pipeline for 79 00:06:55,655 --> 00:07:00,383 diagnoses. And then finally fault diagnosis has been a much, more direct 80 00:07:00,383 --> 00:07:06,095 application of these, of these systems, because here we don't have an issue, of, 81 00:07:06,309 --> 00:07:11,379 of how doctors typically do their diagnostic pipeline. So within the 82 00:07:11,379 --> 00:07:16,948 Window's operating system, there are thousands of these little troubleshooters 83 00:07:16,948 --> 00:07:22,589 that help you diagnose problems with your printer, with, with Excel, with your 84 00:07:22,589 --> 00:07:27,872 email, and each of these has a little Bayesian network inside that answers 85 00:07:27,872 --> 00:07:33,849 probability questions, given observations about what, What the system, the, the, 86 00:07:33,849 --> 00:07:39,695 about the, about the model involving, in this case, for example, the printer. And 87 00:07:39,695 --> 00:07:46,376 there's also a big website out there that does, car repair. And, you put in the make 88 00:07:46,376 --> 00:07:51,994 and model of, and year of the car. And what are the main problems within it. 89 00:07:52,222 --> 00:07:58,372 Figures it out and tells you what to look at, and what the most likely complaints 90 00:07:58,372 --> 00:08:03,784 is. And the reason behind the benefits of this, people don't use Bayesian networks 91 00:08:03,784 --> 00:08:09,097 are cool even though they are, they use this because it provides a very flexible 92 00:08:09,097 --> 00:08:14,154 user interface for the user. You instantiate the evidence of the Bayesian 93 00:08:14,154 --> 00:08:19,146 network out comes the probability, you don't want to answer the question right 94 00:08:19,146 --> 00:08:24,331 now, that's okay, you can answer it later. It's just means that it's an observation 95 00:08:24,331 --> 00:08:29,818 that you didn't get the condition on. And, and then for the designer this type of 96 00:08:29,818 --> 00:08:35,106 system is really easy to design and maintain. Because if for example something 97 00:08:35,106 --> 00:08:39,919 changes a little bit in your printer structure. If you were to design a 98 00:08:39,919 --> 00:08:45,682 standard menu-base system you'd have to go and rebuild the entire tree that asks you 99 00:08:45,682 --> 00:08:51,173 know wh, what is the besides what is the first question to ask, what is the second 100 00:08:51,173 --> 00:08:56,618 question to ask. What is the most likely diagnosis. Here in the Bayesian network, you change 101 00:08:56,618 --> 00:09:01,327 one probability, maybe add an edge, and everything just emerges from that in a 102 00:09:01,327 --> 00:09:06,158 very straight-forward way so it's much more modular and more maintainable than, 103 00:09:06,342 --> 00:09:10,990 than a hardwired menu-based system. And that's what the people who use these 104 00:09:10,990 --> 00:09:16,490 systems will you tell is that, why, that's why they chose this path as opposed to As 105 00:09:16,490 --> 00:09:19,540 opposed to the hard-wired methodology.