The increasing velocity of change puts pressure on organizations to make better data driven decisions. What this means for the enterprise is to have the ability to derive insights from the data faster, and with greater accuracy. This will lead to cognitive/AI or machine learning becoming an essential ingredient in enterprise applications and data science tools for automated insight generation. Data Scientists are now gravitating towards using Machine Learning algorithms over traditional statistical techniques. They realize that these algorithms are better and more accurate in achieving the outcomes they desire.
With enterprises realizing the potential of insight generation from unstructured data; the dominant data science architecture will include purpose built optimized solutions for both structured and unstructured datasets. Yes, you are looking at a deluge of Data lakes!. Also, NLP + ML + AI efforts will result in robust platform approaches to derive potent domain specific insights from these unstructured datasets. Deep Learning and Neural Networks, which extend the power of ML by learning from unstructured data in layers will become essential toolkits in the Data Science community due to their ability to process text, video, IoT ( all forms of unstructured data) and astounding prediction accuracy with these huge datasets.
As companies encompass more data, and figure how to derive insights better from it - this will lead to revenue growth from information based products. Data monetization will grow to be a more important objective for enterprises, and a lot of Data Science effort will go in discovering business value to create data products.
As algorithms take over more of our lives, Data Scientists will be tasked with being responsible for the consequences of the algorithms they create. This will lead to more interpretable models. i.e If the models you are building are designed to be decision support systems for stakeholders, then how do you ensure they understand the way these decisions are made. For example - your models might be building internal decision trees for coming to the answer that you want to predict. Can you expose how your model is making these decisions? Can you ensure that there are no hidden variables, or factors which are predictive of the outcome you want to get to.?
In a 2011 report on big data, the McKinsey Global Institute (MGI) foresaw an extreme shortage of data scientists (140,000 to 190,000 people with “deep analytical skills” in the U.S by 2018.) As more Data Scientists enter this industry – there will be a burgeoning need for ‘Data Translators’ who understand the intersection of Business + Data Science + AI to be able to translate value of work done to executive stakeholders, and the organization as a whole. Data Science after all is useless unless actions are taken to derive value from insights; and there is a great cultural barrier to that happening if the executives do not understand the work of these data scientists well
Five Key Data Science trends in 2018 :
By Afrozy Ara, Head of Data Science Practice at Incedo Inc.