Connect with us

Tech News

Big data: overcoming cherry picking and endless variables

Big data presents businesses of all size with unprecedented insight into before unseen details, but does more data equal more problems? The answer may not be yes or no…

Published

on

big data

big data

Big data presenting big problems?

We have long written about “Big Data” which is defined as large data sets which cannot be managed with simple, common software that captures and processes the data, and is typically consisting of at least dozens of terabytes in a single data set. The challenges of Big Data are, clearly big, and most attention is being paid to the massive amounts of data being generated by social media sites like Facebook and Foursquare.

But it’s not just consumer data that is being called into question. Industry expert Nassim Taleb recently opined on WIRED, “With big data, researchers have brought cherry-picking to an industrial level. Modernity provides too many variables, but too little data per variable. So the spurious relationships grow much, much faster than real information. In other words: Big data may mean more information, but it also means more false information.”

Taleb addresses something that could lead one to think that big data is faulty and bad, but perhaps Taleb is really pointing out the human nature that is still required in some instances of analyzing big data – and most people would not typically question a researcher or their methods, leaving analysis in its youngest phase subjective.

Defining variables is key in big data analysis

Nitin Mayande is the Co-Founder and Chief Scientist at Tellagence which offers the next step in social marketing intelligence through predictive products. Mayande has a PhD in Engineering Management and is a well respected figure in the industry. He tells AGBeat that Taleb’s statement isn’t necessarily wrong, nor is it completely correct.

Mayande noted that Taleb “is right in saying that with big data one can use many many more variables but to say too little data per variable might not be right. By the way when analyzing the data, the efficacy of variables depends upon the frame of reference and the system definition one chooses. If one uses a open system instead of closed system it may not be possible to define variables at all.”

The great irony of big data

Chris Treadaway, CEO and Founder of Polygraph Media which is famous for data-driven analytics said, “To analyze big data, you have to know when you have enough data, know that you’re looking at the right data, and know how and when to draw conclusions from the data using methods developed from statistics theory and data science. That’s the great irony of “big data” – it’s as much of an art as a science, which is why the best efforts are multidisciplinary.”

“Big data can find tremendous hidden relationships,” Treadaway continued, “but you have to make sure your bias isn’t to find conclusions that don’t exist. Bias can cause the situation Taleb describes, and will cause disinformation as he says. If you’re cautious, discerning, and careful, you can make the most of big data. But there are pitfalls for the careless.”

More data is not the answer

Matt Hixson, Co-Founder and CEO of Tellagence stated, “I would agree that more data is not the answer. Most problems need the right data – not an infinite set. I see this a ton in social, people don’t fully understand what is going on so they just keep correlating more and more data.”

Hixson continued, “This is all that most of the big data scientists at Twitter, LinkedIn and Facebook are doing because the don’t know what else to do. The platforms that are generating this data – social networks are a primary source – creating new phenomena that people don’t understand yet.”

Big data remains a treasure trove of information while also presenting challenges as the world attempts to make sense of the tool and how to analyze the infinite amounts of data.

Lani is the Chief Operating Officer at The American Genius and sister news outlet, The Real Daily, and has been named in the Inman 100 Most Influential Real Estate Leaders several times, co-authored a book, co-founded BASHH and Austin Digital Jobs, and is a seasoned business writer and editorialist with a penchant for the irreverent.

Continue Reading
Advertisement
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Tech News

Yell ‘Marco,’ this app makes your phone yell ‘Polo’

(TECH NEWS) New iPhone app helps find your iPhone so you can find your phone to phone a friend. Are you ready to play Marco Polo on land?

Published

on

marco polo iphone voicemail

Do you ever find yourself taking your phone too seriously? Is your phone more of a business tool than anything else? Do you honestly not mind when you lose it?

When you think of your phone, what comes to mind first – April Nardini’s “silent German films,” or Rory Gilmore’s jaunts with the Life and Death Brigade? Consider spicing up your relationship with your phone with role playing! And yes, don’t worry. There’s an app for that.

It’s called Marco Polo, and you may have already guessed what it does, which in my book means it’s a pretty darn good name.

Once you install the app, you can literally shout the word “Marco” into oblivion and your phone will sing out “Polo” at the top of its smart little lungs.

Those who are looking to get themselves and their phones out of deep relationship ruts can endlessly customize the way they address their phone, and choose how it responds.

The customization possibilities seem endless. I suggest terms of endearment: when screamed into the darkness of your early morning bedroom, words like “Dearest!” and “Honeybunch!” are likely to open up a healthy dialogue between you and your phone by making it feel safe and appreciated.

It will then respond in one of thirty preset character voices with the words you’ve instructed it to say. Because phones don’t have free will yet, duh.

The Marco Polo app is extra handy when you don’t have a real live friend around you to call your phone for you, and when you’ve left your phone on silent or very low volume to block out the outside world.

It’ll boost your phone’s volume and light up its screen against its (not-free) will, no matter how carefully you’ve stifled it.

Basically, whenever you’re lonely or stressed and can only find solace in your misplaced phone, this app will be there for you like no one else can be.

The website doesn’t specify how nearby your phone needs to be, or how loudly you need to shout at it, so a little friendly trial and error will get your spicy new relationship up and running.

Marco Polo is only available on Apple devices but it's so worth it.Click To Tweet

Because it definitely isn’t 2018, and most of the entire world definitely doesn’t use Android. Come on, guys.

Continue Reading

Tech News

The semantic argument of the phrase ‘Full Stack’

(TECH NEWS) As the tech industry knows, being able to classify your job qualifications is paramount.

Published

on

lean in coding full-stack

Semantics

A new debate is emerging in the web development world and it’s not about which framework is best, or which language is most marketable.

bar
In fact the debate isn’t a matter of code, it’s a matter of words.

It’s Not Just About Experience Level

“Full Stack Developer” is the title developers both new and old often use to describe themselves. According to a Stack Overflow developer survey touted as the “most comprehensive developer survey conducted” the title is among the top five respondents used to describe themselves.

However, not everyone thinks newer developers should adopt the title.

It would be easy to distill the debate to a matter of experience level, veterans earned the “full stack” title, while newer programmers haven’t. However, there’s way more layers to this debate.

What Exactly is Full Stack

First of all, a simple google search reveals several different definitions of “full stack.” There’s general consensus when it comes to the high-level definition. CodeUp sums up this definition, “The term full stack means developers who are comfortable working with both back-end and front-end technologies.”

When it comes down to the nitty-gritty of what exactly falls under back-end and front-end, there’s some disagreement.

Mastery level also matters, but again there’s disagreement over what’s acceptable. In one camp, are the proficiency pushers who require not only a breadth of understanding, but also a depth of understanding in multiple areas.

In this camp, it’s not just good enough to have exposure to SQL, one must have proficiency in SQL.

In the other camp, are the generalist. They also require a breadth of knowledge, but are happy with a basic familiarity of each stack element. When it comes to debating whether newer developers should adopt the full stack title, the lack of clarity on what full stack means in the first place is a major stumbling block.

Why Full Stack?

Besides clarifying the what behind “full stack” some folks are also clarifying the why. According to Indeed’s job trends, the number of postings and searches matching “full stack developer” on average has trended upwards since 2012 . The title’s popularity causes some to believe that new developers are adopting the title as a buzzword with no real care put into understanding what “full stack” means.

Android Programmer Dan Kim from Basecamp warns, “Just don’t fall back to labeling yourself with a bullshit buzzword that everyone else uses.”

For others, adopting the full stack title is a matter of mindset. As Web developer Christian Maioli over at TechBeacon writes: “To me, a full stack developer is someone who has the curiosity and drive to test the limits of a technology and understand how each piece works generally in various scenarios. Having this mindset will give developers more value and more power in dealing with new situations.”

In both cases, understanding why a new developer adopts the full stack title is connected to understanding whether they’re overselling their skills and how valuable their skills are to a potential employer.

Beyond Job Titles

Finally, this debate about whether new developers should use the “full stack” title brings up the need for alternative methods of measuring proficiency. This need isn’t limited to the web development world, as technology innovates job titles become convoluted.

A job title won’t be the most reliable way to communicate what you bring to a job or what you expect.Click To Tweet

Quantifying what you’ve accomplished in the past, along with what tools you used will be critical in a time where job titles aren’t trusted.

This story was first published here on April 7, 2017.

Continue Reading

Tech News

We’ve all seen job listings for UX writers, but what exactly is UX writing?

(TECH NEWS) We seeing UX writer titles pop up and while UX writing is not technically new, there are new availabilities popping up.

Published

on

writers net neutrality twitter facebook outlook email drag

The work of a UX writer is something you come across everyday. Whether you’re hailing an Uber or browsing Spotify for that one Drake song, your overall user experience is affected by the words you read at each touchpoint.

A UX writer facilitates a smooth interaction between user and product at each of these touchpoints through carefully chosen words.

Some of the most common touchpoints UX writers work on are interface copy, emails and notifications. It doesn’t sound like the most thrilling stuff, but imagine using your favorite apps without all the thoughtful confirmation messages we take for granted. Take Eat24’s food delivery app, instead of a boring loading visual, users get a witty message like “smoking salmon” or “slurping noodles.”

Eat24’s app has UX writing that works because it’s engaging.

Xfinity’s mobile app provides a pleasant user experience by being intuitive. Shows that are available on your phone are clearly labeled under “Available Out of Home.” I’m bummed that Law & Order: SVU isn’t available, but thanks to thoughtful UX writing at least I knew that sad fact ahead of time.

Regardless of where you find a UX writer’s work, there are three traits an effective UX writer must have. Excellent communication skills is a must. The ability to empathize with the user is on almost every job post.

But from my own experience working with UX teams, I’d argue for the ability to advocate as the most important skill.

UX writers may have a very specialized mission, but they typically work within a greater UX design team. In larger companies some UX writers even work with a smaller team of fellow writers. Decisions aren’t made in isolation. You can be the wittiest writer, with a design decision based on obsessive user research, but if you can’t advocate for those decisions then what’s the point?

I mentioned several soft skills, but that doesn’t mean aspiring UX writers can’t benefit from developing a few specific tech skills. While the field doesn’t require a background in web development, UX writers often collaborate with engineering teams. Learning some basic web development principles such as responsive design can help writers create a better user experience across all devices. In a world of rapid prototyping, I’d also suggest learning a few prototyping apps. Several are free to try and super intuitive.

Now that the UX in front of writer no longer intimidates you, go check out ADJ, The American Genius’ Facebook Group for Austin digital job seekers and employers. User centered design isn’t going anywhere and with everyone getting into the automation game, you can expect even more opportunities in UX writing.

Continue Reading

Emerging Stories