Connect with us

Tech News

Big data – buzz word or businesses’ wildest dream?

What exactly is big data? What is a petabyte? Why does it matter to a small business or independent contractor, and is it just a buzz word or is it a marketer’s dream?



What is big data?

“Big Data” is defined as large data sets which cannot be managed with simple, common software that captures and processes the data, and is typically consisting of at least dozens of terabytes in a single data set. The challenges of big data are, well, big. It is described by Gartner analyst, Doug Laney1 as being three-dimensional, i.e. increasing volume (amount of data), velocity (speed of data in/out), and variety (range of data types, sources).

The term is being leveraged in conferences across the globe, but most are talking about the challenges of big data as it pertains to social networking sites like Facebook or location based services like Foursquare, and the idea of the volume of data being collected on every single person and how that data is becoming so large that it is a logistical challenge to manage and harvest with any meaning.

In fact, big data was a popular theme at the recent South by Southwest Conference in Austin, with technologists and marketers bringing their unique backgrounds to the conversation, each addressing the collection of and processing of the unprecedented data being collected, for the first time outside of the government, and the concerns that go along with consumers blindly offering up the data.

How BIG is big data?

Strictly speaking to social data, consider for a moment that in a 60 second period, 23,148 apps are downloaded from the App Store, and 208,333 minutes of Angry Birds is played via smartphone3. Additionally, over 28,000 text messages are sent every second, and the average mobile phone user has 736 pieces of personal data collected every day and service providers store this information for one to five years3.

The University of Nebraska physics department4 has 1.6 petabytes of data – that’s 1.6 million gigabytes in one department at one school. Boeing jet engines can produce 10 terabytes of operational information for every 30 minutes they turn4.

Twitter produces 20 gigabytes of data every day

Twitter serves more than 200 million users who produce over 800 tweets per second, each of which is roughly 200 bytes in size4, so on an average day, this traffic equals over 12 gigabytes, and throughout the Twitter ecosystem, the company produces a total of eight terabytes of data per day, compared to the New York Stock Exchange’s single terabyte of data daily.

Like a Boeing jet records every move, every time you interact with Facebook, it records data. It stores information on who clicks what and not just the name of the person, but that person’s profile information like hobbies, high school, family members, ethnicity, religion, employer, etc. Then, that data connects that one click with all other clicks you make within Facebook or any website with the Facebook plugin. That is a lot of information to store for a few simple clicks, especially given how much more demographic information is recorded and tracked than Twitter which creates 20 times the amount of data in a single day than the NYSE.

What to do with the data?

The big challenge behind the scenes is how to process and manage this volume of information in an era where consumers voluntarily offer thousands of data points through social networks, smartphones and the like. Social data is where the buzz is at conferences, but it is being referred to solely as big data, which clearly is much more complex than just what someone clicked on Facebook.

IT expert, Andrew McCafee recently shared the story5 of an Allstate-sponsored contest wherein a small team of data scientists quickly achieved a 340 percent improvement in Allstate insurance’s ability to predict bodily insurance claims based on car characteristics – without any expertise in insurance or automobiles, and without consulting Allstate’s well paid mathematicians who build and maintain these prediction models.

McAfee asks, “So how can it be that a small team of people who don’t even work for the company was able, in three months, to achieve a 340% improvement over Allstate’s ability to predict bodily injury insurance claims based on car characteristics? And how was the team able to do this while working only with disguised data — without, in other words, knowing the true makes and models of the cars? Welcome to the weird new world big data.”

What this means for business

All of the social data collected in recent years is finally becoming useful for more than buying a Facebook ad. True demographics are now evident, but moreover, real consumer behavior is being studied based on tremendously large amounts of data. If just one simple text message records a dozen pieces of data, imagine the depth of information a Facebook user transmits in a single day, with all of their personal data attached to each move.

Social data (or big data) is a nightmare to manage due to the sheer volume and velocity of transmission, but for businesses, it means a legitimate understanding of consumer behavior, not just what someone shared during a focus study or web poll, but really being able to track and understand how each consumer ticks. What’s next is being able to translate that behavior into an isolated profile of a specific type of buyer – this is the kind of data that marketers’ have been dreaming about for decades.

13D Data Management PDF
2Mobclix study
3How phone carriers track private data
4Information Management study
5Allstate prediction model

Lani is the Chief Operating Officer at The American Genius - she has co-authored a book, co-founded BASHH and Austin Digital Jobs, and is a seasoned business writer and editorialist with a penchant for the irreverent.

Continue Reading


  1. Matthew Hardy

    March 19, 2012 at 6:53 pm

    > consumers blindly offering up the data

    > … to translate that behavior into an isolated profile… marketers’ have been dreaming about for decades.

    Remember Soylent Green? Big data is us. 😉

Leave a Reply

Your email address will not be published. Required fields are marked *

Tech News

Airbnb has blocked 50K+ bookings for being too big during COVID-19

(NEWS) Airbnb has cancelled a huge number of reservations as a security precaution during COVID-19 in the past year or so.




In the last year or so, Airbnb has purposefully prevented at least 50,000 people from making irresponsible reservations on their properties, in many cases blocking those people from the platform itself. This prevention, at least in theory, helped cut down on the number of COVID parties during the pandemic.

According to The Verge, Airbnb’s head of trust and safety communication, Ben Breit, acknowledged blocked reservations in several cities across the United States, including Dallas, San Diego, and New Orleans. Breit confirmed that this response was an attempt to prevent large gatherings and parties during the height of the COVID-19 pandemic during which many areas banned group activities involving more than a few people.

While some requests for reservations were simply denied or “redirected”, many users were blocked from using Airbnb entirely. Airbnb noted that the number of blocked requests outpaced the number of people who were blocked, signifying that some accounts attempted to make more than one reservation before being removed from the platform.
Airbnb reportedly stated that “Instituting a global ban on parties and events is in the best interest of public health” prior to enacting a total ban on rentals at the beginning of 2020, a decision that gave way to the blocks and redirections in the last 12 months.

The evaluation system used to flag problematic reservations is relatively simple, according to Breit: “If you are under the age of 25 and you don’t have a history of positive reviews, we will not allow you to book an entire home listing local to where you live.”

But Airbnb didn’t entirely remove multiple-body listings or large rentals. The Verge reports that flagged users with the aforementioned criteria were still able to book both small rentals in local locations and larger rentals in reasonably distant locations.

Regardless of the optics here, Airbnb’s policy efficacy can’t be ignored. Multiple cities reported comparatively “quiet” holiday seasons–something that may contribute to Airbnb’s decision to extend their policy through the end of this summer.

The hosting company is also offering increased security measures, such as noise detection and a 24-hour hotline, at a discounted rate to property owners.

As both the vaccine gap and the proliferation of the Delta variant of COVID-19 continue to contribute to outbreaks, one can reasonably expect Airbnb to hold to this policy.

Continue Reading

Tech News

TL;DV summarizes video meetings so folks can catch up in quickly *with* context

(TECHNOLOGY) TL;DV makes catching up on video team meetings slightly more tolerable and easily digestable.




2021 was the year of virtual meetings, and while there are some perks associated with remote collaboration (I’m looking at you, pair of work pants that I didn’t have to wear once this year), these meetings often feel exponentially more arduous than their dressed-up counterparts. TL;DV, a consolidation app for Google Meet, looks to give back a bit of your time.

TL;DV (an acronym for “Too Long; Didn’t View”) is a Google Chrome recording extension that helps users specify important sections of meetings for anyone who needs to view them asynchronously. Users can tag specific segments in Google Meet sessions, transcribe audio, and leave notes above tagged sections for timestamp purposes, and the subsequent file can be shared via a host of both Google and third-party apps.

While the extension is only available for Google Meet at the time of writing, the TL;DV team has included a link to a survey for Zoom and MS Teams users on their site, thus implying that the team is looking into expanding into those platforms in the future.

The mission behind TL;DV is, according to the website, to empower users to “control how we spend our precious time” in the interest of combatting FOMO and meeting fatigue. By dramatically shortening the amount of time one must spend perusing a meeting recording, they seem well on their way to doing so.

Of course, the issue of human oversight remains. It seems likely that meeting facilitators will drop the ball here and there while tagging sections of the recording, and employees who miss crucial information in a recorded session are sure to be frustrated in the process–just not as frustrated as they might be if they attended the entire meeting live.

The current (free) version of TL;DV is in Beta, so users will have a three-hour cap on their videos. The development team promises a professional version by the end of 2021, with the added bonus of leaving prior recordings available for free for anyone who used the Beta. This is certainly an extension to keep an eye on–whether or not you’re remaining remote in 2022, virtual conferencing is no doubt here to stay.

Continue Reading

Tech News

Hiding from facial recognition is a booming business

(TECH NEWS) ‘Cloaking’ is the new way to hide your face. Companies are making big money designing cloaking apps that thwart your features by adding a layer of make up, clothing, blurring, and even transforming you into your favorite celebrity.



Facial recognition companies and those who seek to thwart them are currently locked in a grand game of cat and mouse. Though it’s been relentlessly pursued by police, politicians, and technocrats alike, the increasing use of facial recognition technology in public spaces, workplaces, and housing complexes remains a widely unpopular phenomenon.

So it’s no surprise that there is big money to be made in the field of “cloaking,” or dodging facial recognition tech – particularly during COVID times while facial coverings are, literally, in fashion.

Take Fawkes, a cloaking app designed by researchers at the University of Chicago. It is named for Guy Fawkes, the 17th century English revolutionary whose likeness was popularized as a symbol of anonymity, and solidarity in V For Vendetta.

Fawkes works by subtly overlaying a celebrity’s facial information over your selfies at the pixel level. To your friends, the changes will go completely unnoticed, but to an artificial intelligence trying to identify your face, you’d theoretically look just like Beyonce.

Fawkes isn’t available to the general public yet, but if you’re looking for strategies to fly under the radar of facial recognition, don’t fret; it is just one example of the ways in which cloaking has entered the mainstream.

Other forms of cloaking have emerged in the forms of Tik Tok makeup trends, clothes that confuse recognition algorithms, tools that automatically blur identifying features on the face, and much more. Since effective facial recognition relies on having as much information about human faces as possible, cloaking enthusiasts like Ben Zhao, Professor of computer science at the University of Chicago and co-developer of Fawkes, hope to make facial recognition less effective against the rest of the population too. In an interview with The New York Times, Zhao asserts, “our [team’s] goal is to make Clearview [AI] go away.”

For the uninitiated, Clearview AI is a start-up that recently became infamous for scraping billions of public photos from the internet and privately using them to build the database for a law enforcement facial recognition tool.

The CEO of Clearview, Hoan Ton-That, claimed that the tool would only be improved by these workarounds and that in long run, cloaking is futile. If that sounds like supervillain talk, you might see why he’s earned himself a reputation similar to the likes of Martin Shkreli or Ajit Pai with his company’s uniquely aggressive approach to data harvesting.

It all feels like the beginning of a cyberpunk western: a story of man vs. machine. The deck is stacked, the rules are undecided, and the world is watching. But so far, you can rest assured that no algorithm has completely outsmarted our own eyeballs… yet.

Continue Reading

Our Great Partners

American Genius
news neatly in your inbox

Subscribe to our mailing list for news sent straight to your email inbox.

Emerging Stories

Get The American Genius
neatly in your inbox

Subscribe to get business and tech updates, breaking stories, and more!