Home Blog

[ D] Confession as an AI researcher; attempting advice

I have a confession to make.

I was a CS major in college and took very few advanced math or stats courses. Besides basic calculus, linear algebra, and probability 101, I took merely one machine learning class. It was about very concrete SVMs/ decision tree/ probabilistic graphical models that I rarely encounter today.

I joined a machine learning laboratory in college and was mentored by a senior PhD. We actually had a couple of publishings together, though they were nothing but minor architecture changes. Now that I’m in grad school doing AI research full-time, I believed I could continue to get away with zero math and clever lego building. Unfortunately, I fail to produce anything creative. What’s worse, I find it increasingly hard to read some of the latest papers, which probably don’t look complicated at all to math-minded students. The gap in my math/ stats knowledge is taking a hefty toll on my career.

For example, I’ve never heard of the term “Lipschitz” or “Wasserstein distance” before, so I’m unable to digest the Wasserstein GAN paper, let alone invent something like that by myself. Same with f-GAN( https :// arxiv.org/ pdf/ 1606.0070 9. pdf ), and SeLU( https :// arxiv.org/ pdf/ 1706.0251 5. pdf ). I don’t have the slightest clue what the 100 -page SeLU proof is doing. The “Normalizing Flow”( https :// arxiv.org/ pdf/ 1505.0577 0. pdf) paper even involves physics( Langevin Flow, stochastic differential equation) … each word seems to require a semester-long course to master. I don’t even know where to start wrapping my head around.

I’ve thought about potential solutions. The top-down approach is to google each unfamiliar lingo in the paper. That doesn’t work at all because the explanation of 1 unknown points to 3 more unknowns. It’s an exponential tree expansion. The alternative bottom-up approach is to read real analysis, functional analysis, probability hypothesi textbook. I opt a systematic treatment, but …

reading takes a huge amount of hour. I have the next conference deadline to gratify, so I can’t only put aside two months without producing anything. My advisor wouldn’t be happy. but if I don’t read, my mindless lego building will not yield anything publishable for the next conference. What a chicken-and-egg vicious cycle. the “utility density” of reading those 1000 -page textbooks is very low. A plenty of pages are not relevant, but I don’t have an efficient way to sift them out. I understand that some knowledge might be useful some day, but the reward is too sparse to justify my attention budget. The vicious cycle kicks in again. in the ideal world, I can query an oracle with “Langevin flow”. The prophecy would return a listing of pointers,” devoted your current math capability, you should first read chapter 7 of Bishop’s PRML book, and then chapter 10 of information theory, and then chapter 12 of …”. Google is not such an prophecy for my purpose.

Scientists report first detection of gravitational waves produced by colliding neutron stars

Astronomers detect gravitational waves and a gamma-ray burst from two colliding neutron stars. (credit: National Science Foundation/LIGO/Sonoma State University/A. Simonnet)

Scientists reported today (Oct. 16, 2017) the first simultaneous detection of both gravitational waves and light — an astounding collision of two neutron stars.

The discovery was made nearly simultaneously by three gravitational-wave detectors, followed by observations by some 70 ground- and space-based light observatories.

Neutron stars are the smallest, densest stars known to exist and are formed when massive stars explode in supernovas.

MIT | Neutron Stars Collide

As these neutron stars spiraled together, they emitted gravitational waves that were detectable for about 100 seconds. When they collided, a flash of light in the form of gamma rays was emitted and seen on Earth about two seconds after the gravitational waves. In the days and weeks following the smashup, other forms of light, or electromagnetic radiation — including X-ray, ultraviolet, optical, infrared, and radio waves — were detected.

The stars were estimated to be in a range from around 1.1 to 1.6 times the mass of the sun, in the mass range of neutron stars. A neutron star is about 20 kilometers, or 12 miles, in diameter and is so dense that a teaspoon of neutron star material has a mass of about a billion tons.

The initial gamma-ray measurements, combined with the gravitational-wave detection, provide confirmation for Einstein’s general theory of relativity, which predicts that gravitational waves should travel at the speed of light. The observations also reveal signatures of recently synthesized material, including gold and platinum, solving a decades-long mystery of where about half of all elements heavier than iron are produced.

Georgia Tech | The Collision of Two Neutron Stars (audible frequencies start at ~25 seconds)

“This detection has genuinely opened the doors to a new way of doing astrophysics,” said Laura Cadonati, professor of physics at Georgia Tech and deputy spokesperson for the LIGO Scientific Collaboration. I expect it will be remembered as one of the most studied astrophysical events in history.”

In the weeks and months ahead, telescopes around the world will continue to observe the afterglow of the neutron star merger and gather further evidence about various stages of the merger, its interaction with its surroundings, and the processes that produce the heaviest elements in the universe.

The research was published today in Physical Review Letters and in an open-access paper in The Astrophysical Journal Letters.


KurzweilAI has assembled this timeline of the observations from various reports:

  • About 130 million years ago: Two neutron stars are in their final moments of orbiting each other, separated only by about 300 kilometers (200 miles) and gathering speed while closing the distance between them. As the stars spiral faster and closer together, they stretch and distort the surrounding space-time, giving off energy in the form of powerful gravitational waves, before smashing into each other. At the moment of collision, the bulk of the two neutron stars merge into one ultradense object, emitting a “fireball” of gamma rays.
  • Aug. 17, 2017, 12∶41:04 ET: Virgo detector in Pisa, Italy picks up a new strong “chirp” gravitational wave signal, designated GW170817. The LIGO detector in Livingston, Louisiana detects the signal just 22 milliseconds later, then the twin LIGO detector in Hanford, Washington, 3 milliseconds after that. Based on the signal duration (about 100 minutes) and the signal frequencies, scientists at the three facilities conclude it’s likely from neutron stars — not from more massive black holes (as in the previously three gravitational wave detections). And based on the signal strengths and timing between the three detectors, scientists are able to precisely  triangulate the position in the sky.  (The most precise gravitational-wave detection so far.)
  •  1.7 seconds later: NASA’s Fermi Gamma-ray Space Telescope and the European INTEGRAL satellite detect a gamma-ray burst (GRB) lasting nearly 2 seconds from the same general direction of sky. Both the Fermi and LIGO teams quickly alert astronomers around the world to search for an afterglow.
  • Hours later: Armed with these precise coordinates, a handful of observatories around the world starts searching the region of the sky where the signal was thought to originate. A new point of light, resembling a new star, is found by optical telescopes first. Known as a “kilonova,” it’s a phenomenon by which the material that is left over from the neutron star collision, which glows with light, is blown out of the immediate region and far out into space.
  • Days and weeks following: About 70 observatories on the ground and in space observe the event at various longer wavelengths (starting at gamma and then X-ray, ultraviolet, optical, infrared, and ending up at radio wave frequencies).
  •  In the weeks and months ahead: Telescopes around the world will continue to observe the radio-wave afterglow of the neutron star merger and gather further evidence about various stages of the merger, its interaction with its surroundings, and the processes that produce the heaviest elements in the universe.

“Multimessenger” astronomy

Caltech’s David H. Reitze, executive director of the LIGO Laboratory puts the observations in context: “This detection opens the window of a long-awaited ‘multimessenger’ astronomy. It’s the first time that we’ve observed a cataclysmic astrophysical event in both gravitational waves and electromagnetic waves — our cosmic messengers. Gravitational-wave astronomy offers new opportunities to understand the properties of neutron stars in ways that just can’t be achieved with electromagnetic astronomy alone.”

caltech | Variety of Gravitational Waves and a Chirp (audible sound for GW170817 starts ~30 seconds)

Read more: www.kurzweilai.net

Rosa Parks and the power of refusing to move.

60 years ago, on Dec. 1, 1965, Rosa Parks refused to give up her seat to a white rider on a Montgomery bus and sat her way right into the history books. We all know the tale.

Image via Lauren/ Picasaweb.

Many of us have taken plenty of feel-good lessons from it about being brave and taking a stand. Most of the lessons focus on the huge impact of her apparently small action. But there’s also an important life lesson to be learned from the action itself.

Rosa Parks sparked a motion by refusing to move.

Sometimes, choosing to sit still is the most impactful action we can take. Sometimes what starts the movement that we so desperately need is actually our repudiation to be moved.

It seems so counterintuitive. We’ve been taught that to change things, “were supposed to” exert energy, “were supposed to” fighting inertia, and somehow force things to change with our motion by tearing things down with brute force or, in some cases, running the other direction. So how can stillness actually spark radical change?

Photo via Joel Nilsson/ Wikimedia Commons.

I once went to a yoga class where the mantra was “I am like the sunshine. I am big, I am bright, and I will not be moved . ”

It was based on the premise that all the other planets revolve around the sun, the center of our galaxy. I loved this idea and said it to myself every time I needed to feel grounded and resolute, confident that despite the chaos whirling around me, I did not have to move. I could stand peacefully and firm, like the immovable sunshine, in who I was and what I believed.

I held on to that mantra for quite awhile until I discovered that the sun actually does move. It’s simply substantially harder to recognize and watch the movement because of its relation to all the planets spinning around it. In other terms, even when the sun looks like it isn’t moving, it is.

Ready to get deep? Go with me here.

Rosa Parks was the sun that day.

In her refusal to move seats, she appeared to be still even though a huge, important switching truly was taking place. As a result, she forced others to move around her. White bus patrons, police, supporters, society, and ultimately, the law.

Photo via piper6 0/ Pixabay.

It’s clear to see how that lesson relates to activism and social change. Period and time again, from sit-ins at lunch counters and college campuses, to die-ins on the floor of city hall, we’ve watched how the act of being seemingly still and not moving from the scene of injustice can interrupt and ultimately transform unjust systems.

But what if we also applied that principle to our own lives?

So often we believe that in order to stimulate dramatic change, in order to be treated how we deserve to be treated, we have to be the ones to metaphorically move; to change something about ourselves.

We madly move in the face of difficulty, disrespect, or opposition: We cease the job, we relocate, we mitigate our demands, we adjust our appearances, expectations, or approach, we “fall back” to avoid the confrontation.

But if we’re honest, oftentimes our actions are the same thing as moving to the back of the bus. We believe that if “weve been” quiet, if we are accommodating, if we do what is asked of us, if we remove ourselves from the situation entirely, we will either win the respect of the individuals who stand in our style or at the very least, we will make our lives easier.

Ultimately, we do this because we are afraid of the results of being ourselves, standing in our truth, and taking up the space that we deserve.

But what if we finally recognized that the cost of moving is actually greater to our identity and our souls than the cost of refusing to move no matter how scary the immediate consequences is a possibility?

What if the critical behavior change that will win us our liberty is eventually transgressing the pattern of adjusting, accommodating, and moving in the face of opposition?

What if we behaved like the sun? What if by “not moving” we were actually changing not only our own view but everything around us ?

Sounds good, right? But lessons like this are often easier said than done.

Photo by Mark Wilson/ Getty Images.

How do you know when refusing to move is the right action to take?

Well, here are some tips, straight from Rosa’s playbook:

Refuse to move … when you have a plan. Despite the children’s storybook version of events( “Rosa Parks spontaneously decided that she was too tired to move out of her seat! ” ), we now know that her action that day was about as strategic as it gets. She was not the first to refuse her seat, but it had been decided that this was the moment for someone to try again and that she was the right person to do it. The NAACP knew that Rosa’s arrest would be the example that best allow for a successful court case.

You should always think about the impact that standing firm and refusing to move could make and plan for how you are able to deal with the consequences, regardless of which route it turns out.

Refuse to move … when you’ve done everything else and you’re tired. There’s a myth that Rosa Parks was tired after having worked a long day and that her physical fatigue is why she refused to stand. The truth is that she was indeed tired, but not the route most people guess. From her 1992 autobiography “Rosa Parks, My Story”:

“People always say that I didn’t give up my seat because I was tired, but that isn’t true. I was not tired physically, or no more tired than I usually was at the end of a working day. I was not old, although some people have an image of me as being old then. I was forty-two. No, the only tired I was, was tired of giving in.”

Parks’ response to the system of segregation did not begin on the bus that day. She had marched and protested many times before, but on that day, she knew that merely a pure act of defiance would spur the change that needed to take place. The same could be true for you. If you have adjusted and changed and run and objected enough times, refusing to move might just be the ultimate act , not only of defiance, but of freedom.

Refuse to move … when it is morally right. Sitting in her seat wasn’t only a randomly selected act of protest. It was, above all, right. Rosa had principle on her side. And there is no better reason to refuse to move than when principles, values, and morals support your presence and your position.

Photo by Justin Sullivan/ Getty Images.

Rosa Parks’ action will, of course, be remembered forever as one of heroism and will, an act that triggered a turning point in the American civil rights movement.

But it should also be an action that we turn to repeatedly as a reminder of the power of not giving in, of being still.

She showed us that great things can happen when we stay on the bus and refuse to be moved . You, me, and the sunshine, we rise each morning with the same potential and power. And we, too, can change the world.

Read more: www.upworthy.com

Ray Kurzweil + Peter Diamandis: Disruptive Technologies, Mind-Boggling Predictions, and ‘Dangerous Ideas’

Ray Kurzweil and Peter Diamandis presented an Abundance 360 webinar on Friday, October 13 on mind-boggling predictions and transformative (even “dangerous”) ideas.

Read more: www.kurzweilai.net

Holger Czukay, bassist with Can, succumbs aged 79

The man who helped give the German psychedelicists their driving rhythm section was found at his apartment, with the cause of death currently unknown

Holger Czukay, co-founder and bassist with pioneering German boulder band Can, has died aged 79.

He was found by a neighbour at his apartment, converted from Can’s old studio in Weilerswist near Cologne. The cause of death is currently unknown. His wife Ursula had died in July, while his former bandmate Jaki Liebezeit– with whom he generated Can’s fabled driving rhythm section- also succumbed this year, in January.

Can were part of a 1970 s motion rather insensitively dubbed” krautrock” by the British music press, alongside bands including Neu !, Faust and Tangerine Dream, who paired the strident rhythm of rock’n’roll with psychedelic, exploratory new sorts. Following a period of analyse under composer Karlheinz Stockhausen in the early 1960 s, Czukay played on nine of Can’s albums as well as engineering them, including their celebrated Tago Mago and Ege Bamyasi, before leaving in 1977 to go solo.

He went on to collaborate with numerous celebrated musicians- he and Liebezeit played on the Eurythmics‘ debut album, while he recorded the Balearic disco classic Snake Charmer with New York DJ Francois Kevorkian, U2 guitarist The Edge and Public Image Limited’s Jah Wobble, and also made a pair of collaborations with Japan singer David Sylvian.

Czukay is also celebrated for his experiments with sampling before the advent of digital samplers, cutting up and splicing tape into recordings. He also pioneered what he called ” radio paint”, using shortwave radios to record random snippets of sound and pasting them collage-style into recordings;” rhythm boxing” was his description of how he utilized drum machines. Its recent solo album was Eleven Years Innerspace in 2015.

Can partly reformed in April without Czukay- Irmin Schmidt and Malcolm Mooney performed with the London Symphony Orchestra, Sonic Youth guitarist Thurston Moore and others as The Can Project.

Read more: www.theguardian.com

Understanding Variational Autoencoders’ latent loss word

This is a cross post from here since I didn’t know which subreddit is most suited to ask such a question.

I’m trying to understand VAEs latent loss term in their cost function and currently failing at the math behind it. I wrote a stats.se post here, if someone could take a look at it and help me figure out how those functions were obtained that’d be awesome

https :// stats.stackexchange.com/ topics/ 304289/ variational-autoencoder-understanding-the-latent-loss

Read more:

Google merely unveiled 2 new, highly secure messaging apps

As if we didn’t have enough applications to message friends, Google launched two new messaging apps, Duo and Allo at its developer meeting on Wednesday, and they’ll even work on your iPhone.

Allo is a text-based chat app that cooks Google features directly into the app. You can chat with Google Search, ask it pull up appropriate photos in response to messages, share videos, connections, and locations, similar to the features on Google’s new keyboard.

Additionally, machine learning software within the app can understand messages to and from your contacts, and offer up answers. And image recognition tech can recognize objects in photographs and serve up suggestions for what to say back.

Soon, you won’t even need to chats with a person, but automated reactions can chat backward and forward for you.


There are a handful of other quirky features in Allo, like the ability to increase the size of the text called “whisper and holler, ” so you can add emphasis in messages, and “ink, ” that lets you write on photos.

Incognito messaging on Allo has end-to-end encryption to chat privately, receive discreet notifications, and have messages expire, without Google being able to read your messages. Incognito messaging is the only feature of Allo with end-to-end encryption.

Duo, another standalone mobile application, is the video chat companion to Allo. It supposed to be faster and sleeker than other apps on the market, in case you want an alternative to FaceTime, Skype, or even Google’s own Hangouts. It use your telephone number as an identifier and, like Allo, features end-to-end encryption, it’s like almost every other video calling service.


One unique feature called “knock knock” was met with mixed reactions. The app will show you a preview of yourself, and present it to the person you’re calling even before they pick up. It’s going to be harder to ignore calls when you’re greeted with someone’s smiling face before you even answer.

Google said both apps will be available this summer for iOS and Android.

Read more: