Icons of EE: Thomas Bayes

Writing articles for the Vonk has become a rare skill as a human being. With Large Language Models as competition, any time spent writing articles “by hand” seems in vain. As I am essentially an applied mathematical computer scientist of origin, the principles of apparent artificially intelligent mathematical systems are naturally of interest to me. The hype surrounding OpenAI’s ChatGPT led me to follow some courses on machine learning, and the theorem that is at the basis of any was published 260 years ago after Thomas Bayes’ death. This article introduces the Presbyterian preacher that devised the key to estimating algorithms.

Suppose, you are a first year electrical engineering student that was still busy writing their profielwerkstuk just an (eventful) year ago, when you were still an innocent high school kid. During the Kick-In, you met all these very smart people that were in your do-group with which you had a fun time, but got you in doubt that also you will eventually attain that highly esteemed bachelor degree. You know nothing yet about electronics or the complex mathematics the older year students were warning you for, how are you ever going to make it? The estimation that you back then (“prior”) could do was that there was about a 1% chance you will make it and pass all 12 modules in order to get your degree..

So, what are my chances?

But, here you are. You (hopefully) managed to pass module 1 which means it is time to update your estimation of success! Out of the 12 modules that constitute a bachelors degree in Electrical Engineering, you managed to pass at least 1 with a current success rate of 100%! Of course, you could have just had a lucky month, but according to Bayes’ theorem, the chance that you will pass your bachelor degree is described as follows:

This is purely based on chance, assuming all courses of the bachelor have the exact same difficulty, but this is of course not the case. Either way, you can see that as you pass more modules, the chance that you were not just lucky, but that it is likely that you will pass all the modules, improves significantly. If you passed all courses at the end of module 4, there is a very big chance you will make it through the entire bachelor as you have managed to pass already 4/12 modules by then. This is a great advocate for the BSA system, which says that you need to have passed at least 45 out of the 60 EC’s, as it according to Bayes’ theorem ensures that you will manage to pass all the courses in the bachelor. The chance is lower if you haven’t passed all the courses in your first year, but it is still very much possible. I hope this is reassuring to you :)

The preacher who did mathematics

Now, for the introduction of the Reverend Thomas Bayes (1702-1761). Surely, he was never a first year electrical engineering student, so for what reason did he come up with this theorem? Like his father, he was a nonconformist theologian, but he liked to do mathematics on the side. He wrote his friends about his mathematical theories, who got him elected as a Fellow of the Royal Society in 1742. There are no mathematical publications from Bayes himself, but letters written to his friends would be published after his death. He wondered about the age-old question: “Does there exist a God?” and tried to use probability theory to estimate whether it would be likely that one exists.

“Every drop of rain that falls in Sahara Desert says it all: It’s a miracle”

David Hume, a famous Scottish philosopher, argued that because the likelihood that a miracle occurred and the witness was not deceiving or being deceived is very low - and if the miracle through which God would manifest himself are unlikely to have happened, he probably doesn’t exist. But, according to Bayes’ theorem, there is still hope: even though the prior estimation is very low, when there are multiple independent testimonies, at some point the Bayesian estimation of God’s existence will still reach a likely amount (like we saw in your estimation of getting your bachelor degree when you started back in September). The question should be: at how many independent testimonials of miracles does it become undeniable that a God does exist? (or: how many passed modules does it take for you to believe that you can get your degree in the end? ;) )

You can imagine how useful this can be to a computational algorithm that you want to make adaptive to inputs from the physical realm. This is why ChatGPT will sometimes echo misinformation at you: if it is independently mentioned by multiple sources somewhere on the internet, it will assume it to increase in likeliness of truth. Fortunately, they were only using data from up to September 2021 - but with premium membership, this is circumvented, which might lead to undesired misinformation loops. Therefore, it is ever more important to always keep in mind that the information from a person that is knowledgeable in the field and used the scientific method to offer reproducible experiments should always be valued above a majority of opposing views that do not have these credibilities. Perhaps God does not exist, but if you see on Osiris that you passed MOD01, you are 10x more likely to pass the rest of the courses, no matter how good the other students are or how hard the material might seem like at first.

I was inspired by this article to make the bachelor degree analogy. This is a great video if you want to know more about Bayes Inference. If you want to follow courses in machine learning or deep learning, I recommend to follow Machine Learning I (201600070) and Deep Learning: From Theory to Practice (201800177) during your master’s.