Model:

1. A model (no matter how crazy it sounds) is Judged often by How well it explains "Observations"

2. Occam's Razor Principle may be helpful in deciding amongst different models explaining observations. A Model with the least assumptions is best. (Though it is not an absolute criteria) .

3. Fruitfulness : Does the model suggests many new things to try?

4. In most field of knowledge it is necessary to introduce technical notation and jargon only to avoid Vagueness of words

5. Randomness is the negative property. It is the absence of any pattern. Is that not an absence of a pattern is a pattern? (Of course, Hamming rejected this Idea).

6. Randomness is a Mathematical Concept not a Physical One.

7. "If you believe too much you'll never notice the flaws; if you doubt too much you won't get started. It requires a lovely balance."

8. "....all models are wrong, but some are useful"

There are four steps to the Feynman Learning Technique:

1. Choose a concept you want to learn about

2. Pretend you are teaching it to a student in grade 6

3. Identify gaps in your explanation; Go back to the source material, to better understand it.

4. Review and simplify (optional)

Only when you encounter gaps in your knowledge—where you forget something important, are not able to explain it, or simply have trouble thinking of how variables interact—can you really start learning.

“The person who says he knows what he thinks but cannot express it usually does not know what he thinks.”

— Mortimer Adler

https://fs.blog/2012/04/feynman-technique/

I’m very skeptical of doing what is trendy and popular because then you are just playing follow the leader. Everyone jumps into the field all doing more or less the same stuff because that is where the funding is and that is the easiest way to publish papers. In my opinion, this trendiness leads to a massive amount of invested effort but with very few significant results because what everyone is doing is so similar and overlapping. I suppose it is a form of the law of diminishing returns. The big breakthroughs that fundamentally change our understanding come from the people who follow their own path even when everyone else is running in the other direction. Unfortunately, physics like other academic fields usually doesn’t give much support to those who don’t want to play follow the leader.

On Funding:

There are simply so many problems facing not just the US but the entire planet these days ranging from climate change to massive wealth and income inequality in this country. It is unconscionable for tenured academic researchers to earn very generous salaries from their faculty positions and research grants and not be using their abilities to help solve some of these problems. Many are doing just that but one has to wonder how string theorists are contributing to society when even most of the physics community doesn’t understand what they are doing.

On Religion:

The anthropic principle just seems absurd to me, and I wish science and particularly physics was more accepting of religion and faith. They answer completely different questions. Science can explain how things work in the universe and can make predictions about how they will function in the future, but it can’t answer at a fundamental level why the universe is the way it is or how it came to be. Those are the domains of religion and faith. Also, people have felt since as far back as we know a deep connection to something greater than and beyond the universe that we perceive. This transcends culture and society and is present in all religions and forms of spirituality. Physics though discounts the idea that there exists something beyond what we can model with our equations or capture in our experimental data. That though does not mean it is any less real than quantum mechanics or Maxwell’s equations.

I am nevertheless very skeptical of organized religion, which has often been nothing more than a system for a small elite to consolidate power and influence over the masses. I think that one’s faith and connection with God or the universe is deeply individualistic and everyone must follow their own spiritual path. Religious texts and theologians can serve as guides and advisors on one’s path but nothing more. We should all listen to God directly and not to a priest standing at an altar.

Link : https://blogs.scientificamerican.com/cross-check/whats-wrong-with-physics/

Can you explain what survivorship bias is?

Imagine you got a letter in the mail that says, “Hey Katy, I have a new stock-picking trick. And since I know you don't trust me yet, I want to tell you look at Whatever Incorporated tomorrow, and it’s going to go up.”

You say, “Well, I don't know.” But you look at it, and it went up. But anyone can get lucky. So the next week, you get another letter saying, “Tomorrow, I want you to look at Johnson Incorporated, and it’s going to go down.” Now you’re intrigued. You look at Johnson Incorporated, and it does go down. Now you’re waiting for the third letter. It does come, and it’s exactly right.

Now the person says, “If you’re interested in having me as your advisor, you should call me.” Do you see where all this is headed?

This is actually a thing that was run in the 1940s or maybe the 1930s. They sent a bunch of random guesses to 10,000 people. Half the time, they were right. So then, to that half of the people, they sent another bunch of random guesses, which, half the time, were right.

When you start with 10,000, after four guesses, you’ve divided the pool by 16, which is still a pretty big population who now thinks you’re amazing. And what the population is suffering from here is survivorship bias. They’re the set of people who happened to survive, and so now they have this entirely false belief.

Survivorship bias is an error that arises because we look at the data we have but ignore the selection process that led us to have those data. That principle applies in so many places, especially to people like you and me.

.

.

Look at billionaires. No one says, “That’s a person who won a lottery ticket.” People say, “I would love advice from that person.”

So I think survivorship bias really colors how we look at the world, because it leads us to look at these highly selected events and then make inferences and say, “Oh, that manager and that person must be good.”

.

.

Similarly, there was a work done that showed that people who had car accidents were also more likely to have cancer. It was kind of a puzzle until you think, “Wait, who do we measure cancer in?” We don’t measure cancer in everybody. We measure cancer in people who have been tested. And who do we test? We test people who are in hospitals. So someone goes to the hospital for a car accident, and then I do an MRI and find a tumor. And now that leads to car accidents appearing to elevate the level of tumors. So anything that gets you into hospitals raises your “cancer rate,” but that’s not your real cancer rate.

That’s one of my favorite examples, because it really illustrates how even with something like cancer, we’re not actually measuring it without selection bias, because we only measure it in a subset of the population.

How can people avoid falling prey to these kinds of biases?

Look at your life and where you get feedback and ask, “Is that feedback selected, or am I getting unvarnished feedback?”

Full Article @ https://www.scientificamerican.com/article/the-perils-of-survivorship-bias/

Authors Home Page: https://www.chicagobooth.edu/faculty/directory/m/sendhil-mullainathan

1. Information Bottleneck as possible explanation of how deep learning algorithms work:

Full story @ https://www.quantamagazine.org/new-theory-cracks-open-the-black-box-of-deep-learning-20170921/

2. Is it necessary to come up with an explanation for how neural networks make decisions?

a. That said, there are risks to attempting to divine the entrails of a neural network. “With interpretability work, there’s often this worry that maybe you’re fooling yourself,” Olah says. The risk is that we might try to impose visual concepts that are familiar to us or look for easy explanations that make sense.

That’s one reason some figures, including AI pioneer Geoff Hinton, have raised an alarm on relying too much on human interpretation to explain why AI does what it does. Just as humans can’t explain how their brains make decisions, computers run into the same problem. As Hinton put it in a recent interview with WIRED, “If you ask them to explain their decision, you are forcing them to make up a story.”

Full story @ https://www.wired.com/story/inside-black-box-of-neural-network/

b. Interesting Question posted by Hinton on twitter.

Suppose you have cancer and you have to choose between a black box AI surgeon that cannot explain how it works but has a 90% cure rate and a human surgeon with an 80% cure rate. Do you want the AI surgeon to be illegal ?

Quite good Replies:

• • • • Fei-Fei-li: Healthcare is fundamentally about care for humans, from illness to wellbeing. It would be wonderful to see more AI efforts towards supplementing and augmenting our human doctors and nurses so that in the near future, a human surgeon’s success rate is 100% due to AI assistance.

      • Medicine is already full of black-box

By

Andrew Zimmerman Jones

Updated July 03, 2019

In common usage, the words hypothesis, model, theory, and law have different interpretations and are at times used without precision, but in science they have very exact meanings.

Hypothesis

Perhaps the most difficult and intriguing step is the development of a specific, testable hypothesis. A useful hypothesis enables predictions by applying deductive reasoning, often in the form of mathematical analysis. It is a limited statement regarding the cause and effect in a specific situation, which can be tested by experimentation and observation or by statistical analysis of the probabilities from the data obtained. The outcome of the test hypothesis should be currently unknown, so that the results can provide useful data regarding the validity of the hypothesis.

Sometimes a hypothesis is developed that must wait for new knowledge or technology to be testable. The concept of atoms was proposed by the ancient Greeks, who had no means of testing it. Centuries later, when more knowledge became available, the hypothesis gained support and was eventually accepted by the scientific community, though it has had to be amended many times over the year. Atoms are not indivisible, as the Greeks supposed.

Model

A model is used for situations when it is known that the hypothesis has a limitation on its validity. The Bohr model of the atom, for example, depicts electrons circling the atomic nucleus in a fashion similar to planets in the solar system. This model is useful in determining the energies of the quantum states of the electron in the simple hydrogen atom, but it is by no means represents the true nature of the atom. Scientists (and science students) often use such idealized models to get an initial grasp on analyzing complex situations.

Theory and Law

A scientific theory or law represents a hypothesis (or group of related hypotheses) which has been confirmed through repeated testing, almost always conducted over a span of many years. Generally, a theory is an explanation for a set of related phenomena, like the theory of evolution or the big bang theory.

The word "law" is often invoked in reference to a specific mathematical equation that relates the different elements within a theory. Pascal's Law refers an equation that describes differences in pressure based on height. In the overall theory of universal gravitation developed by Sir Isaac Newton, the key equation that describes the gravitational attraction between two objects is called the law of gravity.

These days, physicists rarely apply the word "law" to their ideas. In part, this is because so many of the previous "laws of nature" were found to be not so much laws as guidelines, that work well within certain parameters but not within others.

Scientific Paradigms

Once a scientific theory is established, it is very hard to get the scientific community to discard it. In physics, the concept of ether as a medium for light wave transmission ran into serious opposition in the late 1800s, but it was not disregarded until the early 1900s, when Albert Einstein proposed alternate explanations for the wave nature of light that did not rely upon a medium for transmission.

The science philosopher Thomas Kuhn developed the term scientific paradigm to explain the working set of theories under which science operates. He did extensive work on the scientific revolutions that take place when one paradigm is overturned in favor of a new set of theories. His work suggests that the very nature of science changes when these paradigms are significantly different. The nature of physics prior to relativity and quantum mechanics is fundamentally different from that after their discovery, just as biology prior to Darwin’s Theory of Evolution is fundamentally different from the biology that followed it. The very nature of the inquiry changes.

One consequence of the scientific method is to try to maintain consistency in the inquiry when these revolutions occur and to avoid attempts to overthrow existing paradigms on ideological grounds.

Occam’s Razor

One principle of note in regards to the scientific method is Occam’s Razor (alternately spelled Ockham's Razor), which is named after the 14th century English logician and Franciscan friar William of Ockham. Occam did not create the concept—the work of Thomas Aquinas and even Aristotle referred to some form of it. The name was first attributed to him (to our knowledge) in the 1800s, indicating that he must have espoused the philosophy enough that his name became associated with it.

The Razor is often stated in Latin as:

entia non sunt multiplicanda praeter necessitatem

or, translated to English:

entities should not be multiplied beyond necessity

Occam's Razor indicates that the most simple explanation that fits the available data is the one which is preferable. Assuming that two hypotheses presented have equal predictive power, the one which makes the fewest assumptions and hypothetical entities takes precedence. This appeal to simplicity has been adopted by most of science, and is invoked in this popular quote by Albert Einstein:

Everything should be made as simple as possible, but not simpler.

It is significant to note that Occam's Razor does not prove that the simpler hypothesis is, indeed, the true explanation of how nature behaves. Scientific principles should be as simple as possible, but that's no proof that nature itself is simple.

However, it is generally the case that when a more complex system is at work there is some element of the evidence which doesn't fit the simpler hypothesis, so Occam's Razor is rarely wrong as it deals only with hypotheses of purely equal predictive power. The predictive power is more important than the simplicity.

Edited by Anne Marie Helmenstine, Ph.D.

From : https://www.thoughtco.com/hypothesis-model-theory-and-law-2699066

Abhay Ashtekar is a theoretical physicist and the founder of loop quantum gravity, an increasingly popular branch of physics that attempts to unify quantum mechanics with Albert Einstein’s theory of general relativity (which celebrates its centenary this year). Currently the Director of the Institute for Gravitational Physics and Geometry at Pennsylvania State University, Ashtekar spoke to Nithyanand Rao and Swetamber Das at IIT Madras on October 7, 2015 about his inspirations, his encounters with Subrahmanyan Chandrasekhar and Roger Penrose, work on gravity and cosmology, and his criticisms of string theory.

.

.

I was really fortunate that I got this exposure to three great people, my great teachers: One was my Ph.D. advisor Robert Geroch. Chandra told me that he felt that except for John von Neumann, he has never seen anyone as brilliant as Bob; and it was true. Bob is extremely brilliant. But then something happened and he suddenly stopped working. I don’t know what happened but before that he was totally off-scale. He had such clarity of thinking, such crispness. I learnt by osmosis – the way of thinking, how to think from scratch. He would take us once a week for a pizza dinner or something and hand us a new research paper. Those days, everything came by mail because there was no arXiV or anything. We were supposed to look at the abstract of the paper and try to guess what it was about and how did they did it. That was very good, because you have to start from scratch and you didn’t know much to begin with; and we were just graduate students. He used to put us in this situation – it was like being thrown into the water and being asked to swim.

With Chandra, I got this deeper sense of values – which is about what is “right”, a moral compass about how to be a good scientist and a good human being. A proper sense of values. Chandra was the one who said that I should go to Oxford for my postdoc. I was fortunate again as I had got several offers but Chandra said I should go to Oxford, so I went there.

I went to Roger Penrose. With Penrose also, it was really unique. He was not as brilliant as Bob Geroch was, or as quick. But he had this way of dreaming, looking into the future, groping in the dark and coming up with completely unbelievable ideas. That’s also something that you cannot learn from a book – you see these people in action and you learn. I think with all these three people I learnt things which I could never have learnt from books. Robert Geroch’s clarity, crispness and speed; with Chandra the backbone, the hard and deep stuff which always makes life meaningful; and with Roger Penrose it was a dream-like quality that is so essential for research.

The hardest thing about research is always – I tell this to my students and postdocs – this balance. When you do something and you are in the middle of it, you need – like Chandra said – a certain amount of scientific arrogance. There’s nothing wrong about scientific arrogance. There’s everything wrong about personal arrogance. Scientific arrogance is basically the belief that, yes, I am going to solve this problem. Even if other people have thought about it, it doesn’t matter; I will solve it. You really have to get into it. You want to get into the details, you want to understand the intricate structure which is laid out in front of you, find the missing links. And things that are completely wrong in your thinking and maybe also in other people’s thinking. At that time you just have to be an extreme optimist. You have to believe that it’s going to work and completely disregard scepticism from other people. But then once it is finished, you have to turn around 180 degrees and you have to look at in “cold blood”. Does it even make sense? And then poke every possible hole in it. It’s just the opposite of what you first did. First you make progress, do things; and then be your worst critic.

These two skills are draining. You can go with the first skill quite a bit, but after a while you don’t advance. You need to have this ability of really going back and looking at things critically and seeing the solidity – and poking every hole that anybody else can poke. If you don’t have this solid foundation, you cannot build on it further. You can just do the first things and not go much further. I think that psychologically and mentally this is tiring, to be able to go back and forth.

One has to be able to cope with frustration as well.

"I mean, I can solve many problems even more elegantly than Chandra might have solved. But the skill is to come up with a problem – to come up with the right problems. Things that are going to change the direction. Things that are not going to be only incremental progress but really could make a difference. And that, I think, is not easy. That is what distinguishes great scientists from good scientists – the ability to really spot this, what is really worth working on. "

Because even if you have a good idea, if you don’t put it in the right way it doesn’t have the same impact as somebody else who might have a lesser idea but puts it in the right framework and makes interconnections

Even for a technical paper, the title and the abstract are really important even if you may think “what’s the big deal?” But those things are important. Somehow those are the skills that you don’t always learn. Maybe because your advisors never tell you. But it’s important. The number of people who’d actually look at that paper would depend on how you write these things.

Article at : https://thewire.in/science/good-scientists-solve-problems-but-great-scientists-know-whats-worth-solving

https://medium.com/@michael.bronstein/deriving-convolution-from-first-principles-4ff124888028