Skip to main content

Chapter 2 - Neddle/Noodle


That was a long list of exercises! The most interesting was surely Buffon's needle and noodle, which I couldn't solve in the proposed time (assuming 1 hour for exercises level 3). After 1 hour I went first to Wikipedia to get some hint before actually reading MacKay's answer. From there I found the integral geometry formulation and the rest was easy.

Life in high-dimensional spaces was also quite interesting although I had encountered this idea before when learning about the curse of dimensionality. One thing that helps me think about it is the difference between having uniformly distributed points in a sphere and having points spread in the sphere so that the deviation from the mean distance between points is very small. Putting it more clearly, think about how could you spread points in a circle (I tried triangle first) so that the distance between a point and its closest neighbour is almost the same for every point? Solving and implementing this gave me a better intuition.

Another exercise I couldn't solve on time was proving the decomposability of the entropy function. I could solve only the first part: Decomposing it into two bits. This was an exercise level 2, but just part I took me more than half an hour. In my defense, though, I could solve an exercise level 3 (proving that the entropy is always smaller or equal to the cardinality of the ensemble's alphabet) in less than one hour. Finally I should mention the Poissonville exercise that took me a while to really understand. One thing that helped was thinking that the overcrowded buses don't have to hold the same number of passengers as the non-overcrowded ones.

Overall it took me two weeks to go through Chapter 2 and I'm beginning to think that this project will take about twice my predict time, that i,s it will probably take me one year and a half to finish.


 Picture: By François-Hubert Drouais - Musée Buffon à Montbard, Public Domain, https://commons.wikimedia.org/w/index.php?curid=308910

Comments

Post a Comment

Popular posts from this blog

Jester in the Dark: Encoding Knowledge into Inferences - Part I

You are dining at the castle hall and the king announces he is going to give the jester's his yearly payment. He points at a small adjacent room, where there are 3 small equal chests. The first with 2 gold coins; the second, 1 gold and 1 silver coin; the third, 2 silver coins. The king's serfs go inside, shuffle the chests, lock them, put out the candles, and summon the court jester to the hall. The buffoon is told to put his hand in a bag, grab 1 chest key, walk into the dark room and get 1 coin out of the matching chest. He brings the coin back and it is a gold one. The king takes the coin from his hand and says aloud: - If I send this idiot back there to get the other coin out of the same chest, how likely it is to be gold? The jester interrupts: - Please do send me, sire! I'd likely be a happy idiot, with one more gold coin. The king angrily points out to the jester's face : - Folly! I'll have you beheaded for stupidity. This gold coin could ha...
Post a Comment
Read more

Slaying The Mythical p

Finally I have come out of the introductory chapters forest. In mission 3.15, MacKay summons us to slay the mythical p-value. The weapons we carry are simple, but sharp and the insights look very promising. This mission will lead us to confront the p-value and its meaning when we are comparing two different models. In this post we are going to carefully go through the exercise solution and insights. First we are going to recalculate the statistician conclusion and then move on to MacKay's approach of comparing two models by how likely is the data given each of them. In a way, this post is also intended as containing part of the essence of chapters 1 and 3. Before reading on I suggest you work on the exercise yourself (for 15 minutes, even if you can't solve it): ‘If the coin were unbiased the chance of getting a result as extreme as that would be less than 7%’ We interpret this sentence as meaning that 7% is the probability of a fair coin getting 140 or mo...
Post a Comment
Read more

First Week, First Chapter

I just finished my first week and want to reflect a bit on why I started this. There are two main reasons: Deepen and broaden my knowledge of Information Theory; and motivate people who might hesitate on taking up the challenge or feel too old to start learning or revisiting a field. The book is very exciting; Mackay's writing is clear and flows nicely, introducing a concept, diving into it, and then out to generalise. His approach is to make you feel as if you are constructing and checking the concepts alongside him. His magic is that the presentation feels both casual and principled, like if you were sitting together with a very smart friend who is explaining you things. I ended this week at page 21, so my current rate is 3 pages per day since I will not be working the weekend. At the current rate I would finish the book in about 6 months. Mackay's prediction for the time required for each exercise worked quite well for me although it took me more time tha...
Post a Comment
Read more