Connect with us

Tech News

Big data: overcoming cherry picking and endless variables

Big data presents businesses of all size with unprecedented insight into before unseen details, but does more data equal more problems? The answer may not be yes or no…

Published

on

big data

big data

Big data presenting big problems?

We have long written about “Big Data” which is defined as large data sets which cannot be managed with simple, common software that captures and processes the data, and is typically consisting of at least dozens of terabytes in a single data set. The challenges of Big Data are, clearly big, and most attention is being paid to the massive amounts of data being generated by social media sites like Facebook and Foursquare.

But it’s not just consumer data that is being called into question. Industry expert Nassim Taleb recently opined on WIRED, “With big data, researchers have brought cherry-picking to an industrial level. Modernity provides too many variables, but too little data per variable. So the spurious relationships grow much, much faster than real information. In other words: Big data may mean more information, but it also means more false information.”

Taleb addresses something that could lead one to think that big data is faulty and bad, but perhaps Taleb is really pointing out the human nature that is still required in some instances of analyzing big data – and most people would not typically question a researcher or their methods, leaving analysis in its youngest phase subjective.

Defining variables is key in big data analysis

Nitin Mayande is the Co-Founder and Chief Scientist at Tellagence which offers the next step in social marketing intelligence through predictive products. Mayande has a PhD in Engineering Management and is a well respected figure in the industry. He tells AGBeat that Taleb’s statement isn’t necessarily wrong, nor is it completely correct.

Mayande noted that Taleb “is right in saying that with big data one can use many many more variables but to say too little data per variable might not be right. By the way when analyzing the data, the efficacy of variables depends upon the frame of reference and the system definition one chooses. If one uses a open system instead of closed system it may not be possible to define variables at all.”

The great irony of big data

Chris Treadaway, CEO and Founder of Polygraph Media which is famous for data-driven analytics said, “To analyze big data, you have to know when you have enough data, know that you’re looking at the right data, and know how and when to draw conclusions from the data using methods developed from statistics theory and data science. That’s the great irony of “big data” – it’s as much of an art as a science, which is why the best efforts are multidisciplinary.”

“Big data can find tremendous hidden relationships,” Treadaway continued, “but you have to make sure your bias isn’t to find conclusions that don’t exist. Bias can cause the situation Taleb describes, and will cause disinformation as he says. If you’re cautious, discerning, and careful, you can make the most of big data. But there are pitfalls for the careless.”

More data is not the answer

Matt Hixson, Co-Founder and CEO of Tellagence stated, “I would agree that more data is not the answer. Most problems need the right data – not an infinite set. I see this a ton in social, people don’t fully understand what is going on so they just keep correlating more and more data.”

Hixson continued, “This is all that most of the big data scientists at Twitter, LinkedIn and Facebook are doing because the don’t know what else to do. The platforms that are generating this data – social networks are a primary source – creating new phenomena that people don’t understand yet.”

Big data remains a treasure trove of information while also presenting challenges as the world attempts to make sense of the tool and how to analyze the infinite amounts of data.

Tech News

Career consultants help job seekers beat AI robot interviews

(TECH NEWS) With the growth of artificial intelligence conducting the job screening, consultants in South Korea have come up with an innovative response.

Published

on

job screening by robot

When it comes to resume screenings, women and people of color are regularly passed over, even if they have the exact same resume as a man. In order to give everyone a fair try, we need a system that’s less biased. With the cool, calculating depictions of artificial intelligence in modern media, it’s tempting to say that AI could help us solve our resume screening woes. After all, nothing says unbiased like a machine…right?

Wrong.

I mean, if you need an example of what can go wrong with AI, look no further than Microsoft’s Tay, which went from making banal conversation to spouting racist and misogynistic nonsense in less than 24 hours. Not exactly the ideal.

Sure, Tay was learning from Twitter, which is a hotbed of cruelty and conflict, but the thing is, professional software isn’t always much better. Google’s software has been caught offering biased translations (assuming, for example, if you wrote “engineer” you were referring to a man) and Amazon has been called out for using job screening software that was biased against women.

And that’s just part of what could go wrong with AI scanning your resume. After all, even if gender and race are accounted for (which, again, all bets are off), you’d better bet there are other things – like specific phrases – that these machines are on the lookout for.

So, how do you stand out when it’s a machine, not a human, judging your work? Consultants in South Korea have a solution: teach people how to work around the bots. This includes anything from resume work to learning what facial expressions are ideal for filmed interviews.

It helps that many companies use the same software to do screening. Instead of trying to prepare to impress a wide variety of humans, if someone knew the right tricks for handling an AI system, they could potentially put in much less work. For example, maybe one human interviewer likes big smiles, while the other is put off by them. The AI system, on the other hand, won’t waver from company to company.

Granted, this solution isn’t foolproof either. Not every business uses the same program to scan applicants, for instance. Plus, this tech is still in its relative infancy – a program could easily be in flux as requirements are tweaked. Who knows, maybe someday we’ll actually have application software that can more accurately serve as a judge of applicant quality.

In the meantime, there’s always AI interview classes.

Continue Reading

Tech News

Google chrome: The anti-cookie monster in 2022

(TECH NEWS) If you are tired of third party cookies trying to grab every bit of data about you, google has heard and responded with their new updates.

Published

on

3rd party cookies

Google has announced the end of third-party tracking cookies on its Chrome browser within the next two years in an effort to grant users better means of security and privacy. With third-party cookies having been relied upon by advertising and social media networks, this move will undoubtedly have ramifications on the digital ad sector.

Google’s announcement was made in a blog post by Chrome engineering director, Justin Schuh. This follows Google’s Privacy Sandbox launch back in August, an initiative meant to brainstorm ideas concerning behavioral advertising online without using third-party cookies.

Chrome is currently the most popular browser, comprising of 64% of the global browser market. Additionally, Google has staked out its role as the world’s largest online ad company with countless partners and intermediaries. This change and any others made by Google will affect this army of partnerships.

This comes in the wake of rising popularity for anti-tracking features on web browsers across the board. Safari and Firefox have both launched updates (Intelligent Tracking Prevention for Safari and the Enhanced Tracking Prevention for Firefox) with Microsoft having recently released the new Edge browser which automatically utilizes tracking prevention. These changes have rocked share prices for ad tech companies since last year.

The two-year grace period before Chrome goes cookie-less has given the ad and media industries time to absorb the shock and develop plans of action. The transition has soften the blow, demonstrating Google’s willingness to keep positive working relations with ad partnerships. Although users can look forward to better privacy protection and choice over how their data is used, Google has made it clear it’s trying to keep balance in the web ecosystems which will likely mean compromises for everyone involved.

Chrome’s SameSite cookie update will launch in February, requiring publishers and ad tech vendors to label third-party cookies that can be used elsewhere on the web.

Continue Reading

Tech News

Computer vision helps AI create a recipe from just a photo

(TECH NEWS) It’s so hard to find the right recipe for that beautiful meal you saw on tv or online. Well computer vision helps AI recreate it from a picture!

Published

on

computer vision recreates recipe

Ever seen at a photo of a delicious looking meal on Instagram and wondered how the heck to make that? Now there’s an AI for that, kind of.

Facebook’s AI research lab has been developing a system that can analyze a photo of food and then create a recipe. So, is Facebook trying to take on all the food bloggers of the world now too?

Well, not exactly, the AI is part of an ongoing effort to teach AI how to see and then understand the visual world. Food is just a fun and challenging training exercise. They have been referring to it as “inverse cooking.”

According to Facebook, “The “inverse cooking” system uses computer vision, technology that extracts information from digital images and videos to give computers a high level of understanding of the visual world,”

The concept of computer vision isn’t new. Computer vision is the guiding force behind mobile apps that can identify something just by snapping a picture. If you’ve ever taken a photo of your credit card on an app instead of typing out all the numbers, then you’ve seen computer vision in action.

Facebook researchers insist that this is no ordinary computer vision because their system uses two networks to arrive at the solution, therefore increasing accuracy. According to Facebook research scientist Michal Drozdzal, the system works by dividing the problem into two parts. A neutral network works to identify ingredients that are visible in the image, while the second network pulls a recipe from a kind of database.

These two networks have been the key to researcher’s success with more complicated dishes where you can’t necessarily see every ingredient. Of course, the tech team hasn’t stepped foot in the kitchen yet, so the jury is still out.

This sounds neat and all, but why should you care if the computer is learning how to cook?

Research projects like this one carry AI technology a long way. As the AI gets smarter and expands its limits, researchers are able to conceptualize new ways to put the technology to use in our everyday lives. For now, AI like this is saving you the trouble of typing out your entire credit card number, but someday it could analyze images on a much grander scale.

Continue Reading
Advertisement

Our Great Partners

The
American Genius
news neatly in your inbox

Subscribe to our mailing list for news sent straight to your email inbox.

Emerging Stories

Get The American Genius
neatly in your inbox

Subscribe to get business and tech updates, breaking stories, and more!