Sunday, December 30, 2012
Saturday, December 29, 2012
By STEVE LOHR
Published: December 29, 2012
IT was the bold title of a conference this month at the Massachusetts Institute of Technology, and of a widely read article in The Harvard Business Review last October: “Big Data: The Management Revolution.”
Andrew McAfee, principal research scientist at the M.I.T. Center for Digital Business, led off the conference by saying that Big Data would be “the next big chapter of our business history.” Next on stage was Erik Brynjolfsson, a professor and director of the M.I.T. center and a co-author of the article with Dr. McAfee. Big Data, said Professor Brynjolfsson, will “replace ideas, paradigms, organizations and ways of thinking about the world.”
These drumroll claims rest on the premise that data like Web-browsing trails, sensor signals, GPS tracking, and social network messages will open the door to measuring and monitoring people and machines as never before. And by setting clever computer algorithms loose on the data troves, you can predict behavior of all kinds: shopping, dating and voting, for example.
The results, according to technologists and business executives, will be a smarter world, with more efficient companies, better-served consumers and superior decisions guided by data and analysis.
I’ve written about what is now being called Big Data a fair bit over the years, and I think it’s a powerful tool and an unstoppable trend. But a year-end column, I thought, might be a time for reflection, questions and qualms about this technology.
The quest to draw useful insights from business measurements is nothing new. Big Data is a descendant of Frederick Winslow Taylor’s “scientific management” of more than a century ago. Taylor’s instrument of measurement was the stopwatch, timing and monitoring a worker’s every movement. Taylor and his acolytes used these time-and-motion studies to redesign work for maximum efficiency. The excesses of this approach would become satirical grist for Charlie Chaplin’s “Modern Times.” The enthusiasm for quantitative methods has waxed and waned ever since.
Big Data proponents point to the Internet for examples of triumphant data businesses, notably Google. But many of the Big Data techniques of math modeling, predictive algorithms and artificial intelligence software were first widely applied on Wall Street.
At the M.I.T. conference, a panel was asked to cite examples of big failures in Big Data. No one could really think of any. Soon after, though, Roberto Rigobon could barely contain himself as he took to the stage. Mr. Rigobon, a professor at M.I.T.’s Sloan School of Management, said that the financial crisis certainly humbled the data hounds. “Hedge funds failed all over the world,” he said.
THE problem is that a math model, like a metaphor, is a simplification. This type of modeling came out of the sciences, where the behavior of particles in a fluid, for example, is predictable according to the laws of physics.
In so many Big Data applications, a math model attaches a crisp number to human behavior, interests and preferences. The peril of that approach, as in finance, was the subject of a recent book by Emanuel Derman, a former quant at Goldman Sachs and now a professor at Columbia University. Its title is “Models. Behaving. Badly.”
Claudia Perlich, chief scientist at Media6Degrees, an online ad-targeting start-up in New York, puts the problem this way: “You can fool yourself with data like you can’t with anything else. I fear a Big Data bubble.”
The bubble that concerns Ms. Perlich is not so much a surge of investment, with new companies forming and then failing in large numbers. That’s capitalism, she says. She is worried about a rush of people calling themselves “data scientists,” doing poor work and giving the field a bad name.
Indeed, Big Data does seem to be facing a work-force bottleneck.
“We can’t grow the skills fast enough,” says Ms. Perlich, who formerly worked for I.B.M. Watson Labs and is an adjunct professor at the Stern School of Business at New York University.
A report last year by the McKinsey Global Institute, the research arm of the consulting firm, projected that the United States needed 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more data-literate managers, whether retrained or hired.
Thomas H. Davenport, a visiting professor at the Harvard Business School, is writing a book called “Keeping Up With the Quants” to help managers cope with the Big Data challenge. A major part of managing Big Data projects, he says, is asking the right questions: How do you define the problem? What data do you need? Where does it come from? What are the assumptions behind the model that the data is fed into? How is the model different from reality?
Society might be well served if the model makers pondered the ethical dimensions of their work as well as studying the math, according to Rachel Schutt, a senior statistician at Google Research.
“Models do not just predict, but they can make things happen,” says Ms. Schutt, who taught a data science course this year at Columbia. “That’s not discussed generally in our field.”
Models can create what data scientists call a behavioral loop. A person feeds in data, which is collected by an algorithm that then presents the user with choices, thus steering behavior.
Consider Facebook. You put personal data on your Facebook page, and Facebook’s software tracks your clicks and your searches on the site. Then, algorithms sift through that data to present you with “friend” suggestions.
Understandably, the increasing use of software that microscopically tracks and monitors online behavior has raised privacy worries. Will Big Data usher in a digital surveillance state, mainly serving corporate interests?
Personally, my bigger concern is that the algorithms that are shaping my digital world are too simple-minded, rather than too smart. That was a theme of a book by Eli Pariser, titled“The Filter Bubble: What the Internet Is Hiding From You.”
It’s encouraging that thoughtful data scientists like Ms. Perlich and Ms. Schutt recognize the limits and shortcomings of the Big Data technology that they are building. Listening to the data is important, they say, but so is experience and intuition. After all, what is intuition at its best but large amounts of data of all kinds filtered through a human brain rather than a math model?
At the M.I.T. conference, Ms. Schutt was asked what makes a good data scientist. Obviously, she replied, the requirements include computer science and math skills, but you also want someone who has a deep, wide-ranging curiosity, is innovative and is guided by experience as well as data.
“I don’t worship the machine,” she said.
A version of this article appeared in print on December 30, 2012, on page BU3 of the New York edition with the headline: Sure, Big Data Is Great. But So Is Intuition..
Be afraid. Be very afraid. "They" have rocket launchers. Give me a break. This is a M136 AT4 rocket launcher tube. The AT4 is a one-use weapon. This fearsome tube is more useless than a spent cartridge. You can't reload it or insert another rocket (if you could get one). They can be purchased at Army and Navy stores, on the net...any venue selling military surplus. The police should know this. Credibility is damaged when the suggestion to the public is that live rocket launchers are in private hands.
The coolest and most influential leftist in Europe tells Salon he battles depression -- and those who worship him
"On Friday morning, the Senate renewed the FISA Amendments Act (PDF), which allows for warrantless electronic eavesdropping, for an additional five years. The act, which was originally passed by Congress in 2008, allows law enforcement agencies to access private communications as long as one participant in the communications could reasonably be believed to be outside the United States. This law has been the subject ofa federal lawsuit, and was argued before the Supreme Court recently. 'The legislation does not require the government to identify the target or facility to be monitored. It can begin surveillance a week before making the request, and the surveillance can continue during the appeals process if, in a rare case, the secret FISA court rejects the surveillance application. The court’s rulings are not public.'"
A major threat to the U.S. aircraft carriers in the region December 29, 2012Posted by Richard Clements
For the third time in 7 years (first one being in 2005, second earlier in 2012) several websites in China (link in Chinese) are reporting that China and Russia have agreed for Beijing to buy the production line for the Tupolev Tu-22M3 bomber at a cost of 1.5 billion USD.
Once in service with the Chinese Naval Air Forces the Tu-22M3 will be known as the “H-10″.
The deal struck with Russia comes with 36 aircraft (and engines): an initial batch of 12 followed by a second batch of 24 aircraft are thought to be on order.
The Tu-22 will be employed in the maritime attack role and will be used to attack targets from low level (to avoid radar detection).
Image credit: Wiki
The Tu-22 is a Soviet supersonic, swing-wing, long-range strategic and maritime strike bomber. It was developed during the Cold War and it is among the farthest things to a moder stealth bomber. However, it was upgraded, it will get updated with (indigenous?) systems and, with a range of about 6,800 kilometers and a payload of 24,000 kg, it is still considered a significant threat to many latest generations weapon systems.
Especially if the deal with Russia includes the Raduga Kh-22(AS-4 ‘Kitchen’) long-range anti-ship missile.
The deal could represent a significant change in the strategic balance in the region.
The Tu-22 bombers will give China another tool to pursue the area denial strategy in the South China Sea and the Pacific theatre; a fast platform to launch cruise missiles, conventional or nuclear weapons in various regional war scenarios.
In other words, a brand new threat to the U.S. Navy in the region.
Written with David Cenciotti
Facebook recently fixed a bug that would have let criminals turn on a person’s webcam and record them without their knowledge, according to Bloomberg. The vulnerability was found by Indian research firm XY Security.
The hole that affected both Windows and Mac machines was reported to Facebook in July and shut down soon thereafter. Facebook spokesperson Fred Wolens confirmed to Bloomberg that the bug had not affected anyone in the billion-person social network.
Wolens explained that the bug only could have affected those who have previously gave Facebook permission to access that computer’s webcam. A criminal could then post a “malicious page” which would prompt the user to activate the webcam, which would start the recording process. The video could only be published if the user then went back to that page and deactivated the web cam, according to Wolens.
Seems like a farfetched attack process, but companies are right to be sensitive to any matters associated with the webcam. Stealing video of a person without their consent or knowledge brings concerns to a whole new level. It seems Facebook agrees and paid the researchers $2,500.
The social network participates in a bug bounty program, similar to its competitor Google. The program allows anyone registered to poke around Facebook and find holes in the company’s code or code from external programs it may use that could lead to a security incident. The idea is to catch them with white hat hackers before the black hats take advantage of the situation.
Webcam photo via Shutterstock
Friday, December 28, 2012
Marc Compeau, Contributor
We used to stop at newsstands, grab the paper to catch up on the world – and take a moment to get an update on the life of the familiar face that sold us the daily news. We used to run to the video store on Friday night, grab a new release for the family and catch up with neighbors while we waited in line. Board games with the family filled Saturday nights and a drive-in movie was a treat that the kids bragged about on Monday morning at school.
Today it’s cell phones and on-line games, Google a movie and download it immediately. Smart phone notifications tell us immediately if something happens, the morning paper is old news.
Instant gratification is convenient and technology is progress that we embrace; why would anyone want anything less? Once you have a smart phone you can’t imagine living without, but are we sacrificing something in exchange for faster, smarter, newer?
While I waited for new tires to be installed this weekend I had a 45-minute conversation – with a stranger – and it was awesome. I learned new things, had a story to tell my family at home and the time passed quickly. But I had to tuck my phone away to do it and that meant no Facebook updates, no Twitter feeds, no Angry Birds, just conversation. And it felt great.
Life is busy for small business owners – you must keep up with the competition. You are constantly asked to find a new way to get you products and services to your customer, faster and cheaper, because that’s what we need to do to make them happy, right? Of course, your small business needs to integrate technology into your offerings; it is a survival necessity. But the advantage that so many small businesses have – the one most often overlooked – is the direct relationships you have with your customers. Relationships can’t be made using technology; relationships come from conversations and experiences and relationships lead to loyalty.
Put down your phone, look up from the monitor and say hello. You might be surprised what happens next.
Mon Dec 24, 2012 at 07:47 AM PST
“Everyone wants to think of education as an equalizer — the place where upward mobility gets started,” said Greg J. Duncan, an economist at the University of California, Irvine. “But on virtually every measure we have, the gaps between high- and low-income kids are widening. It’s very disheartening.” [...]The New York Times' Jason DeParle reports on the struggles of three girls, best friends from Galveston, Texas, to get through college despite their families' low incomes, highlighting many of the challenges such students face. Being at the very top of your class, having taken advantage of every college preparation program and advanced class available to you, being driven and hard-working and smart, isn't necessarily enough if you land in a college environment where most other students had more advanced classes that better prepared them for college work, had tutors and help from their parents and counselors to help them choose the college best suited to them. No matter how hard-working you are, it puts you at a disadvantage if you're working not just to get As but to feed yourself. And then there's the question of just paying for college. Through it all, middle-class students have the advantage not just of parents who can pay more money, but who can advocate for them more effectively, as the sociologist Annette Lareau explains in the article, drawing from her fabulous book Unequal Childhoods.
Thirty years ago, there was a 31 percentage point difference between the share of prosperous and poor Americans who earned bachelor’s degrees, according to Martha J. Bailey and Susan M. Dynarski of the University of Michigan. Now the gap is 45 points.
While both groups improved their odds of finishing college, the affluent improved much more, widening their sizable lead.
So, for instance, one of the students DeParle follows accumulated extra debt because:
Angelica reported that her mother made $35,000 a year and paid about half of that in rent. With her housing costs so high, Emory assumed the family had extra money and assigned Mrs. Lady an income of $51,000. But Mrs. Lady was not hiding money. She was paying inflated post-hurricane rent with the help of Federal disaster aid, a detail Angelica had inadvertently omitted.Angelica was struggling with a new environment, more challenging classes than she'd had in high school, and how to pay for an expensive college and she was trying to do it on her own. Unwilling to saddle her mother—who works at Walmart—with loans, she had her boyfriend co-sign them. She didn't have people to go through her financial aid forms with her to be sure she wasn't missing anything. And it's not because she and the people around her weren't bright or hard-working. They just didn't have the resources or cultural tools to help her make this leap. That's how it is for too many students in our incredibly unequal economy and education system.
By counting money the family did not have, Emory not only increased the amount it expected Angelica to pay in addition to her financial aid. It also disqualified her from most of the school’s touted program of debt relief. Under the Emory Advantage plan the school replaces loans with grants for families making less than $50,000 a year. Moving Angelica just over the threshold placed her in a less-generous tier and forced her to borrow an additional $15,000 before she could qualify. The mistake will add years to her repayment plan.
She discovered what had happened only recently, after allowing a reporter to review her file with Emory officials.
If you look at the stories of individuals, there will be a bewildering array of stories like Angelica's and those of her friends, one of whom is on track to finish a four-year public college after five years, with $44,000 in loans, and one of whom completed an associate degree. But the basic point is this: When the graduation gap between affluent people and low-income people is growing, and especially when rich kids with below-average test scores are more likely to graduate than poor kids with above-average test scores, something is wrong. The American dream is broken.