Connect with us

Tech News

Big data: overcoming cherry picking and endless variables

Big data presents businesses of all size with unprecedented insight into before unseen details, but does more data equal more problems? The answer may not be yes or no…

Published

on

big data

big data

Big data presenting big problems?

We have long written about “Big Data” which is defined as large data sets which cannot be managed with simple, common software that captures and processes the data, and is typically consisting of at least dozens of terabytes in a single data set. The challenges of Big Data are, clearly big, and most attention is being paid to the massive amounts of data being generated by social media sites like Facebook and Foursquare.

But it’s not just consumer data that is being called into question. Industry expert Nassim Taleb recently opined on WIRED, “With big data, researchers have brought cherry-picking to an industrial level. Modernity provides too many variables, but too little data per variable. So the spurious relationships grow much, much faster than real information. In other words: Big data may mean more information, but it also means more false information.”

Taleb addresses something that could lead one to think that big data is faulty and bad, but perhaps Taleb is really pointing out the human nature that is still required in some instances of analyzing big data – and most people would not typically question a researcher or their methods, leaving analysis in its youngest phase subjective.

Defining variables is key in big data analysis

Nitin Mayande is the Co-Founder and Chief Scientist at Tellagence which offers the next step in social marketing intelligence through predictive products. Mayande has a PhD in Engineering Management and is a well respected figure in the industry. He tells AGBeat that Taleb’s statement isn’t necessarily wrong, nor is it completely correct.

Mayande noted that Taleb “is right in saying that with big data one can use many many more variables but to say too little data per variable might not be right. By the way when analyzing the data, the efficacy of variables depends upon the frame of reference and the system definition one chooses. If one uses a open system instead of closed system it may not be possible to define variables at all.”

The great irony of big data

Chris Treadaway, CEO and Founder of Polygraph Media which is famous for data-driven analytics said, “To analyze big data, you have to know when you have enough data, know that you’re looking at the right data, and know how and when to draw conclusions from the data using methods developed from statistics theory and data science. That’s the great irony of “big data” – it’s as much of an art as a science, which is why the best efforts are multidisciplinary.”

“Big data can find tremendous hidden relationships,” Treadaway continued, “but you have to make sure your bias isn’t to find conclusions that don’t exist. Bias can cause the situation Taleb describes, and will cause disinformation as he says. If you’re cautious, discerning, and careful, you can make the most of big data. But there are pitfalls for the careless.”

More data is not the answer

Matt Hixson, Co-Founder and CEO of Tellagence stated, “I would agree that more data is not the answer. Most problems need the right data – not an infinite set. I see this a ton in social, people don’t fully understand what is going on so they just keep correlating more and more data.”

Hixson continued, “This is all that most of the big data scientists at Twitter, LinkedIn and Facebook are doing because the don’t know what else to do. The platforms that are generating this data – social networks are a primary source – creating new phenomena that people don’t understand yet.”

Big data remains a treasure trove of information while also presenting challenges as the world attempts to make sense of the tool and how to analyze the infinite amounts of data.

Tech News

AI technology is using facial recognition to hire the “right” people

(TECH NEWS) Artificial intelligence (AI) technology has made its way into the hiring process and while the intentions are good, I vote we proceed with extreme caution.

Published

on

AI technology facial recognition

Artificial intelligence technology has made its way into the hiring process and while the intentions are good, I vote we proceed with extreme caution.

A UK based consumer goods giant, Unilever, is just one of several UK companies who have begun using AI technology to sort through initial job candidates. The goal of this technology is to increase the number of candidates whom a company can interview at the initial stages of the hiring process and to improve response time for those candidates.

The AI, developed by American company Hirevue, analyzes a candidate’s language, tone, and facial expression during a video interview. Hirevue insists that their product is different from traditional facial recognition technologies because it analyzes far more data points.

Hirevue’s chief technology officer, Loren Larsen, says, “We get about 25,000 data points from 15 minutes of video per candidate. The text, the audio and the video come together to give us a very clear analysis and rich data set of how someone is responding, the emotions and cognitions they go through.”
This data is then used to rank candidates on a scale of 1 to 100 against a database of traits identified in previously successful candidates.

There are two main flaws to this system. First, unless this AI technology is pulling from a huge diverse data pool it could be unintentionally discriminating against people without even being aware of it. Human bias is not as easy to remove from the equation as AI proponents would have you believe.

As an example, how does this AI handle people who are disabled or whose facial expressions that read differently than the general population, such as people with Down Syndrome or those who have survived traumatic facial injuries?

Second, seeking to hire someone who possess the same qualities as the person who was previously successful at a role is shortsighted. There are many ways to accomplish the same task with above average results. Companies who adopt this low-risk mentality could be missing out on great opportunities long-term. You will never know what actually works best if you don’t try.

The big question here is whether or not AI technology is ready to influence the job market on this scale.

Continue Reading

Tech News

The ‘move fast and break things’ trend is finally over

(TECH NEWS) Time is running out for this decade — and for a popular Big Tech phrase responsible for a lot of collateral damage. What’s next?

Published

on

big tech move fast break stuff

Time is running out for the decade. With less than 20 days left, it’s got us reflecting on the journeys of different economic sectors in the United States. And no industry has had a more tumultuous time of it than Big Tech.

A lot has changed in ten years. For starters, Americans have become increasingly disillusioned with Silicon Valley. The Pew Research Center found that only 50 percent of Americans believe technology firms have a positive effect on the country. That statistic is not too bad on its own, but that’s down 21 percent from only four years ago. Gallup found in 2019 that 48 percent of Americans also want more regulations on Big Tech. And The New York Times called the 2010s as “the decade Big Tech lost its way”.

Maybe that’s why big wigs at these tech firms have been quietly ditching a concept that was their Golden Rule in the early part of the decade: Move Fast and Break Things.

This concept is a modern take on the adage “you can’t make an omelet without breaking a few eggs.” For most of these firms, any innovation justified some of the collateral damage within its wake. And this scrappy “build it now and worry about it later” philosophy was a favorite of not just Facebook and Twitter, but also of many venture capital firms too.

But not anymore. Outlets from Forbes to HBR are saying this doesn’t work for Big Tech in the 2020s. Here are some reasons why it’s over.

Stability

The Move Fast and Break Things manta encouraged devs to push their coding changes to go live and let the chips fall where they may. But bugs pile up. Enter technical debt.

“Technical debt happens every time you do things that might get you closer to your goal now but create problems that you’ll have to fix later,” said The Quantified VC in an article on Medium. “As you move fast and break things, you will certainly accumulate technical debt.”

If enough technical debt comes into play, any new line of code could be the thing that topples a firm like a house of cards. And now that the consumer is used to tech in their daily routines, interruptions in service are extremely bad news for everyone.

As Mark Zuckerburg himself said it: “When you build something that you don’t have to fix 10 times, you can move forward on top of what you’ve built.”

Trust

To get back some of the trust that has ebbed from Big Tech over the years, firms can’t just keep with the Move Fast and Break Things status quo.

“The public will continue to grow weary of perceived abuses by tech companies, and will favor businesses that address economic, social, and environmental problems,” said Hemant Taneja in his article for Harvard Business Review. “Minimum viable products must be replaced by minimum virtuous products that … build in guards against potential harms.”

It’s not about chasing the bottom dollar at the cost of the consumer. Losing trust will hurt any company if left unchecked for long.

Innovation

There’s a cap on advancement in our current technological state. It’s called Moore’s Law. And we’re rapidly approaching the theoretical limits of it.

“When you understand the fundamental technology that underlies a product or service, you can move quickly, trying out nearly endless permutations until you arrive at an optimized solution. That’s often far more effective than a more planned, deliberate approach,” said Greg Satell in his article for HBR.

Soon enough, Big Tech will be in relatively new waters with quantum computing, biofeedback and AI. There’s no way to move as fast as these technology firms have in the past. And even if they could, should they?

Big Tech has experienced major growing pains since the dawn of our new Millenium. And now that some firms are entering their 20s, there’s a choice to be made. Continue to grow up or keep using an idea that’s worn out it’s welcome with the consumer and that has no guarantee will work with future technologies.

Maybe that’s why Facebook’s motto is now “Move Fast with Stable Infrastructure.”

Continue Reading

Tech News

Computer vision helps AI create a recipe from just a photo

(TECH NEWS) It’s so hard to find the right recipe for that beautiful meal you saw on tv or online. Well computer vision helps AI recreate it from a picture!

Published

on

computer vision recreates recipe

Ever seen at a photo of a delicious looking meal on Instagram and wondered how the heck to make that? Now there’s an AI for that, kind of.

Facebook’s AI research lab has been developing a system that can analyze a photo of food and then create a recipe. So, is Facebook trying to take on all the food bloggers of the world now too?

Well, not exactly, the AI is part of an ongoing effort to teach AI how to see and then understand the visual world. Food is just a fun and challenging training exercise. They have been referring to it as “inverse cooking.”

According to Facebook, “The “inverse cooking” system uses computer vision, technology that extracts information from digital images and videos to give computers a high level of understanding of the visual world,”

The concept of computer vision isn’t new. Computer vision is the guiding force behind mobile apps that can identify something just by snapping a picture. If you’ve ever taken a photo of your credit card on an app instead of typing out all the numbers, then you’ve seen computer vision in action.

Facebook researchers insist that this is no ordinary computer vision because their system uses two networks to arrive at the solution, therefore increasing accuracy. According to Facebook research scientist Michal Drozdzal, the system works by dividing the problem into two parts. A neutral network works to identify ingredients that are visible in the image, while the second network pulls a recipe from a kind of database.

These two networks have been the key to researcher’s success with more complicated dishes where you can’t necessarily see every ingredient. Of course, the tech team hasn’t stepped foot in the kitchen yet, so the jury is still out.

This sounds neat and all, but why should you care if the computer is learning how to cook?

Research projects like this one carry AI technology a long way. As the AI gets smarter and expands its limits, researchers are able to conceptualize new ways to put the technology to use in our everyday lives. For now, AI like this is saving you the trouble of typing out your entire credit card number, but someday it could analyze images on a much grander scale.

Continue Reading
Advertisement

Our Great Partners

The
American Genius
news neatly in your inbox

Subscribe to our mailing list for news sent straight to your email inbox.

Emerging Stories

Get The American Genius
neatly in your inbox

Subscribe to get business and tech updates, breaking stories, and more!