A recent report paints a compelling picture of the growing need for investment in India’s social sector and reveals how India’s philanthropic community is poised to position the country as a hub for social innovation The Indian social sector spending over the last five years experienced a robust annual growth of 13% and currently stands at approximately INR 23 lakh crore ($280 billion) in FY2023 (8.3% of GDP). Public spending constitutes most of the social expenditure (95%), per the India Philanthropy Report 2024 from Bain & Company. Yet, India still falls behind NITI Aayog’s estimated spending requirements (13% of GDP) stipulated in line with the 17 UN Sustainable Development Goals (SDG) commitments by 2030. These include eradication of poverty, quality education and healthcare, gender equality, and climate action, among others. While the Indian donor base is broadening, corporate social responsibility (CSR)and HNI(HighNet-worth Individual) or Affluent donations are growing moderately. Private philanthropy grew 10% in FY2023, to INR 1.2 lakh crore ($15 billion). This faster growth (vs. FY2018–2023’s 5% annual growth) was driven by growth in family philanthropy (15%) and retail (12%), states the report. However, segments like CSR and (HNI)/ affluent donations grew moderately at 7%, despite an expanding donor base, according to the report. Data indicates that HNIs and affluent individuals have a higher propensity to give than UHNIs (more than 0.7% of net worth vs. 0.1% for UHNIs). Hence, there is potential to unlock a significant upside in donations from this segment with the surge in the Indian economy and capital markets. There has been a notable increase in corporate givers, as evidenced by the proportion of companies complying with the CSR mandate (2% of profits), which increased from approximately 30% in FY2018 to more than 60% in FY2022. Additionally, the share of non-BSE 200 companies participating in CSR initiatives rose from 50% in FY2018 to 59% in FY2022. CSR spending, however, grew moderately at 7% in FY2023. Under family philanthropy, the above 60% growth in Ultra-High Net-worth Individual (UHNI) donations was driven by concurrent donors. CSR has seen a significant surge in compliance, with more companies adhering to the mandated 2% contribution requirement. Regulatory amendments have prioritised transparency and accountability, leading to improved reporting and increased CSR spending. In FY2023, CSR spending is estimated at INR 28,000 crore, with its share of domestic private giving increasing to 30% from 22% in FY2018. The report highlights a shift in the types and durations of CSR projects, with a growing focus on multi-year initiatives aimed at driving sustainable impact. Companies are increasingly prioritising environmental and sustainability initiatives, complementing the emphasis on ESG (Environmental, Social, and Governance) strategies. Family Philanthropy and Retail Giving: Unlocking the Potential Family philanthropy emerges as a key driver of growth in India’s private funding landscape, with a projected CAGR of 12-25% during FY2023–2028. The report also underscores the potential of retail giving, which has seen moderate-high growth driven by increased donations to NGOs and community initiatives. To unlock the full potential of HNI and affluent donors, the report suggests addressing barriers such as lack of trust and transparency in the sector. Professionals are also proactively engaging in philanthropy, motivated by personal reasons and a desire to strengthen the ecosystem. Women Philanthropists: Reshaping Narratives and Driving Impact Women philanthropists are playing a significant role in shaping the philanthropic landscape in India. The report highlights their emphasis on adopting a GEDI (Gender, Equity, Diversity, and Inclusion) lens in their giving approach, which is often correlated to their lived experiences in the Indian context. Funding narratives are being redefined by women philanthropists and this is transforming philanthropic institutions as well. Women donors are tackling complex societal challenges such as caste-based discrimination, social inequity, and intersectional mental health. Their contributions are not only driving impact but also shifting power dynamics and fostering societal healing. Collaborative Philanthropy: Strengthening the Ecosystem The report notes a five-fold increase in the number of collaboratives established yearly since 2020, with a focus on addressing intricate challenges through collective action. Intermediaries are playing a crucial role in anchoring these collaboratives, ensuring a structured and neutral environment. However, collaborative philanthropy faces several hurdles, including insufficient long-term funding, trust barriers, and operational inefficiencies. Addressing these concerns through transparent communication, unrestricted long-term funding, and strong governance structures can unlock the full potential of collaborative giving. A Call to Action: Investing in India’s Social Sector The report paints a compelling picture of the growing need for investment in India’s social sector. The deficit in social sector funding could rise to about INR 15 lakh crore by FY2028, underscoring the urgency for strategic and sustained giving. Philanthropists and corporate donors are urged to make big bets and long-term commitments, focusing on intersectional approaches that prioritise GEDI and climate action. By collaborating with local and international partners, India’s philanthropic community can create a positive global impact and position the country as a hub for social innovation. Acknowledgement: India Philanthropy Report 2024 | Bain & Company Know more about the syllabus and placement record of our Top Ranked Data Science Course in Kolkata, Data Science course in Bangalore, Data Science course in Hyderabad, and Data Science course in Chennai.
Growing Clout of Data Science – Part I
The US Bureau of Labor Statistics forecasts that data scientist roles will remain among the fastest-growing jobs in 2024. Even without much hype or pronounced fanfare, professions built around data science are steadily growing in clout Peter Sondergaard, former Senior Vice President at Gartner, had once memorably stated, “Information is the oil of the 21st century, and analytics is the combustion engine.”In this age of digital transformation, data – the building blocks of information –is powering global innovation. With all the connected devices and smart tech out there, and the ever-growing Internet of Things, we’re generating data like never before – practically ushering in a data-driven revolution. And with sophisticated data analysis techniques, from machine learning to predictive modelling, we are now able to extract some serious insights from all that information. It’s a total game-changer, reshaping how businesses operate and how we lead our lives. The ability to harness the power of data has become the decisive competitive edge, as organisations across sectors strive to maximise the value ofinformation. A Data-Driven Future In the rapidly evolving digital landscape, data science is emerging as a fundamental driving force that is poised to transform every major industry sector. Gone are the days when data analysis was confined to specialised teams and siloed departments. Today, data science is becoming an integral part of strategic decision-making, operational optimisation, and innovative product development in organisations of all sizes and across all verticals. As we peer into the future, it is clear that the influence of data science will only continue to grow, revolutionising the way companies operate, compete, and deliver value to their customers. Only a few years ago, data science was a niche domain about which the average person was quite oblivious. The scenario has changed now, and the increasing importance of data in all aspects of our everyday transactions has led to a specialised demand for data science professionals across industries. The World Economic Forum’s Future of Jobs 2023 report projects a 40% higher demand for AI/ML professionals by 2027, while a 30–35% growth in demand is projected for data analysts, scientists, engineers, BI analysts, big data and database roles. Even without much hype or pronounced fanfare, professions built around data science are steadily growing in clout, although skillsets and job profiles in this domain keep transforming. The US Bureau of Labor Statistics forecasts that data scientist roles will remain among the fastest-growing jobs in 2024, with a 35% projected increase in job openings between 2022 and 2032. Most encouraging is the fact that more and more recruiters are looking for core Data Science degrees to fill up these roles. A Data-Driven Future In the rapidly evolving digital landscape, data science is emerging as a fundamental driving force that is poised to transform every major industry sector. Gone are the days when data analysis was confined to specialised teams and siloed departments. Today, data science is becoming an integral part of strategic decision-making, operational optimisation, and innovative product development in organisations of all sizes and across all verticals. As we peer into the future, it is clear that the influence of data science will only continue to grow, revolutionising the way companies operate, compete, and deliver value to their customers. Only a few years ago, data science was a niche domain about which the average person was quite oblivious. The scenario has changed now, and the increasing importance of data in all aspects of our everyday transactions has led to a specialised demand for data science professionals across industries. The World Economic Forum’s Future of Jobs 2023 report projects a 40% higher demand for AI/ML professionals by 2027, while a 30–35% growth in demand is projected for data analysts, scientists, engineers, BI analysts, big data and database roles. Even without much hype or pronounced fanfare, professions built around data science are steadily growing in clout, although skillsets and job profiles in this domain keep transforming. The US Bureau of Labor Statistics forecasts that data scientist roles will remain among the fastest-growing jobs in 2024, with a 35% projected increase in job openings between 2022 and 2032. Most encouraging is the fact that more and more recruiters are looking for core Data Science degrees to fill up these roles. Figure 1: Academic requirements sought for Data Science jobs; Source: www.365datascience.com Redefining Efficiency and Productivity One of the most tangible ways in which data science is transforming industries is through enhanced operational efficiency and productivity. By applying advanced analytics, machine learning, and predictive modelling to massive datasets, organisations are able to uncover hidden patterns, identify bottlenecks, and make data-driven decisions that drive measurable improvements. By transforming raw data into actionable insights, organisations across industries are able to operate with greater agility, efficiency, and competitiveness. But the influence of data science extends far beyond optimising existing processes. Perhaps even more significantly, data-driven insights are fuelling groundbreaking innovations that are disrupting established industry norms. Powering Innovation and Disruption In manufacturing, for example, data science is enabling smart factories to optimise production workflows, predict equipment failures, and minimise waste and downtime. Retail businesses are leveraging customer data to personalise the shopping experience, optimise inventory and supply chains, and make more accurate sales forecasts. Even in the public sector, data science is being used to enhance resource allocation, improve the delivery of social services, and make cities smarter and more habitable. In the healthcare sector, for instance, the application of data science to health data, is leading to breakthroughs in personalised medicine. Pharmaceutical companies are leveraging analytical tools to fast-track long-drawn R&D procedures in drug development. Banking and financial services are overhauling risk management, trading pattern analytics, and automation. Even traditional industries like agriculture is getting a total facelift through precision farming and predictive analytics – thus being more resilient to the whims of unpredictable weather. The common thread across these diverse examples is the ability of data science to challenge existing assumptions, and pave the way for disruptive solutions. Figure 2: Current industry distribution of Data
Financial Analysis using GPT-4
The landscape of financial analysis is undergoing a seismic shift. Recent research from the University of Chicago demonstrates that large language models (LLMs), specifically OpenAI’s GPT-4, can conduct financial statement analysis with an accuracy that rivals – and often surpasses – that of professional human analysts. This breakthrough has profound implications for the future of financial decision-making. The Chicago Study: LLMs Versus Human Analysts Researchers at the University of Chicago in a groundbreaking study titled “Financial Statement Analysis with Large Language Models,” wanted to determine if a large language model (LLM) can perform financial statement analysis similar to a professional human analyst – a task that demands critical thinking, reasoning, and judgment. In the study, GPT-4 was provided with standardised, anonymous financial statements and instructed to analyse them to predict the direction of future earnings. A significant innovation was also the use of “chain-of-thought” prompts, which guided GPT-4 to emulate the analytical process of a financial analyst. According to the researchers, this approach helped the model perform intuitive reasoning and pattern recognition, capabilities that stem from its vast knowledge base and understanding of business concepts. These carefully-curated prompts were expected to help the AI identify trends, compute ratios, and synthesise information to form sharper predictions. Remarkably, even without any narrative or industry-specific information, the LLM successfully outperformed financial analysts in predicting earnings changes and did better even in situations where analysts typically struggle – such as in areas where they may tend to show bias or disagreement. Surprisingly, GPT-4’s performance matches or exceeds that of specialised machine learning models, such as ANNs (Artificial Neural Networks) trained for earnings predictions and considerably higher than the typical 53-57% accuracy range of human analysts. Perhaps as importantly, the LLM’s predictions did not stem from its training memory; instead, it synthesised, based on the thoughtful prompts, useful narrative insights about a company’s future performance as well. Trading strategies based on GPT-4’s predictions also yielded a higher Sharpe ratio and alphas – used to analyse company performance in trading strategies – than those based on other models. Overcoming Challenges in Numerical Analysis and the Path Forward Despite these impressive results, the study also highlighted the traditional challenges LLMs face with numerical analysis. “One of the most challenging domains for a language model is the numerical domain,” said Alex Kim, one of the study’s co-authors. “While LLMs are effective at textual tasks, their understanding of numbers typically comes from the narrative context, and they lack the deep numerical reasoning or the flexibility of a human mind.” Some experts have expressed caution, noting that the ‘ANN’ model used as a benchmark in the study may not represent the cutting edge of quantitative finance, noted VentureBeat.The remarkable capacity of a general-purpose language model to rival and even outperform specialised machine learning models,and human financial experts as well, underscores their disruptive transformative potential within the financesector. The implications of this research are vast. As AI technology continues to advance, the role of the financial analyst is poised to evolve. While human expertise and judgment remain invaluable, powerful tools like GPT-4 can significantly augment and streamline the work of analysts. While the study’s results are certainly indicative of LLMs’ potential to democratise financial information processing and assist in decision-making, the full extent of their impact on human decision-making in financial markets remains to be explored. As we move forward, the integration of AI in financial analysis will likely grow, offering new opportunities and challenges. The future promises interplay between human expertise and artificial intelligence, heralding a new era in financial analysis that leverages the strengths of both to drive better decision-making. References: Read “Financial Statement Analysis with Large Language Models” (University of Chicago) here. The University of Chicago researchers have also developed an interactive web application to showcase GPT-4’s capabilities, inviting curious readers to explore its potential. However, they caution that the model’s accuracy should be independently verified before widespread adoption. Find it here. Know more about the syllabus and placement record of our Top Ranked Data Science Course in Kolkata, Data Science course in Bangalore, Data Science course in Hyderabad, and Data Science course in Chennai.
AI Revolutionises the Workplace
A comprehensive report produced by Microsoft and LinkedIn affirms that the challenge and opportunity for leaders now is to steer their organisations through significant technological shift and ensuring they leverage AI for transformative business outcomes As tech professionals face job losses, organisations are on the other hand finding it difficult to fill certain roles. According to the 2024 Work Trend Index, a comprehensive report produced by Microsoft and LinkedIn, the labour market is set to shift again – with AI playing a major role. Surprisingly, despite a general worry about job loss, leaders report a talent shortage for key roles. With a growing number of employees’ looking at a career move, recruiting managers say AI aptitude could rival experience as a desirable trait. Over the past eight years, leaders had been rushing to acquire technical AI talent, with hiring up 323% during that period. Now their focus is on non-technical talent with AI aptitude – such as skills to use generative AI tools like ChatGPT and Copilot effectively. The report mentions that a majority (55%) of leaders is concerned about the availability of enough talent to fill such roles going forward. These leaders sit across functions, but the number jumps to 60% or higher for those in cybersecurity, engineering, and creative design. Meanwhile, professionals are looking: While some professionals worry AI will replace their job (45%), about the same share (46%) say they’re considering quitting in the year ahead – higher than the 40% who said the same ahead of 2021’s Great Reshuffle. And, in the US, LinkedIn studies show a 14% increase in job applications per role since last fall, with 85% of professionals considering a new job this year. Source: 2024 Work Trend Index (www.microsoft.com) The report unveils transformative shifts in the global workplace due to the pervasive adoption of Artificial Intelligence (AI). Based on surveys of 31,000 employees across 31 countries, it highlights a world where 75% of knowledge workers now use AI, marking a near doubling of usage within just six months. The burgeoning integration of AI at work is not only reshaping job roles and enhancing productivity but also raising the bar for AI skills across industries. The digital workplace has seen numerous innovations over the years, but none are perhaps as impactful as the current wave of AI adoption. AI’s ability to augment human capabilities and streamline complex tasks is leading to significant shifts in work patterns, job roles, and even employee expectations. The 2024 Work Trend Index provides critical insights into how AI is being integrated into the workplace, the challenges this poses, and the opportunities it creates for businesses and employees alike. Rapid Adoption and Employee Initiative One of the most striking findings from the report is the rapid pace at which AI tools are being adopted by knowledge workers worldwide. Approximately 75% of employees now use some form of generative AI in their daily tasks, a significant increase attributed largely to the technology’s ability to save time and enhance focus on critical tasks. Importantly, the report reveals that 78% of AI users are bringing their own AI tools to work, demonstrating a proactive approach by employees to leverage new technologies, often ahead of official adoption by their employers. The Impact of AI on Work Dynamics Integrating AI in an existing system comes with its own challenges. The report notes a discrepancy between the eagerness of employees to adopt AI and the preparedness of organisations to manage this shift. While 79% of leaders acknowledge the necessity of AI for staying competitive, 60% admit their organisations lack a clear plan to implement AI effectively. This gap highlights the need for strategic leadership to harness AI’s potential fully. AI is also transforming job roles and expectations. The report indicates that AI skills are becoming a prerequisite in the job market, with 66% of leaders stating they would not hire a candidate without AI competencies. Moreover, AI is breaking the traditional career ceiling, offering new opportunities for those who adapt quickly to its demands. Business Transformation through AI As businesses move past initial AI experimentation, the focus is shifting towards sustainable integration that drives business transformation. Successful organisations are those that apply AI to enhance productivity, manage costs, and deliver superior customer value, thereby gaining a competitive edge in their markets. The labour market is simultaneously undergoing significant changes, influenced by AI adoption. Despite concerns about job displacement, the report identifies a hidden talent shortage, particularly for roles requiring AI proficiency. This presents a paradox where the demand for skilled AI professionals is increasing even as automation becomes more prevalent. Recommendations for Organisations To capitalise on AI’s transformative potential, organisations need to develop clear strategies that encompass both technological adoption and workforce development. The report suggests several approaches: Strategic AI implementation: Identify key business processes that can benefit from AI and integrate tools to enhance efficiency and decision-making. Leadership and vision: Cultivate a leadership mindset that embraces AI, with clear communication and strategic planning to guide AI adoption. Training and development: Invest in continuous learning and development programs to equip employees with the necessary AI skills. This not only enhances productivity but also aids in employee retention and satisfaction. A Pivotal Point in AI at Work The report believes that the world has arrived at a pivotal moment for AI at work. Looking back at the pre-PC times we now wonder how we ever managed, and that sense of amazement will return again when we reflect on the pre-AI days a few years down the line. AI is already assisting us to become more creative and productive, and also providing a competitive advantage to job seekers. Indeed, it is set to transform every facet of the workplace. The more challenging aspect of this technological disruption is converting experimentation to tangible business value – and companies that can tackle this challenge will lead the market. In the current situation, fortune will definitely favour the bold! The 2024 Work Trend Index from Microsoft and LinkedIn offers
The AI Trust Gap
The black box nature of many AI modelsoften prevents people fromtrusting intelligent machines.As we move forward, it’s crucial to prioritise the development of AI systems that are not only powerful but also trustworthy and transparent The tech industry has poured tens of billions into developing new AI models, with leading players like OpenAI seeking trillions more in investment. The goal is to steadily demonstrate better AI performance and close the gap between human and machine capabilities. Interest in AI, building since last year, will push a 10% increase in data centre system spending this year, driving worldwide IT spending to $5.06 trillion, said John-David Lovelock, distinguished vice president analyst at Gartner. NASSCOM reports that nearly 90% of companies plan to boost spending on top digital technology priorities in 2024, a significant increase from past years. It is predicted that over the next 6-12 months, the majority of investments will focus on AI and machine learning technologiessuch as generative AI, intelligent automation, and big data analytics. But there is another critical gap that deserves equal, if not higher, priority – the AI trust gap. This gap refers to the persistent risks, both real and perceived, that prevent people from being willing to entrust machines with tasks that would otherwise be handled by qualified humans. The Trust Gap Concerns According to a recent HBR article, this trust gap spans a range of concerns, from disinformation and safety/security risks to ethical issues, bias, and the black box nature of many AI systems. And it’s a major problem – a 2023 survey found that between 37.8% and 51.4% of AI and machine learning experts placed at least a 10% probability on scenarios as dire as human extinction resulting from unsafe AI. The trust issues go even deeper. 85% of internet users now worry about their inability to spot fake content online, a serious problem given the rise of AI-aided deepfakes in recent elections from Bangladesh to Moldova. Businesses are also grappling with a litany of AI risks, with the majority of experts seeing a high likelihood of AI systems being “jailbroken” to follow illegal commands. The problem is that no matter how advanced AI becomes, the trust gap will remain a permanent fixture. Efforts to improve transparency, enforce ethical guidelines, and mitigate biases will only provide partial remedies. The black box nature of many AI models, the context-dependent nature of ethical dilemmas, and the inevitability of some biases mean the trust gap will persist. This has major implications. First, AI adopters – consumers, businesses, policymakers – will always have to traverse this trust gap. Second, companies must invest in understanding and addressing the specific risks driving mistrust in their applications. A recent Cornell study, for example, found that a New York law requiring employers to audit their AI hiring tools for bias was largely toothless. And third, pairing humans with AI will be essential, as we will always need humans to guide us through the trust gap. As one expert noted, “the industry has spent tens of billions in creating AI products, such as Microsoft Copilot. It’s time to also invest in the human alongside: the pilot.” The lesson is clear. The industry has spent billions creating AI products, but it’s time to also invest heavily in the human component. Preparing humans to recognise the causes of the AI trust gap, accept its permanence, and learn how to effectively oversee and complement AI systems is crucial. Only then can we realise the full potential of AI while maintaining public trust. A Path to Trust and Interconnectedness As we stand at the threshold of 2024, the world is poised to witness a significant shift in the way we interact with artificial intelligence (AI). The advancements in large language models (LLMs) like GPT have opened doors to new possibilities, but it’s crucial that we address the trust issues that arise from these powerful tools. The integration of LLMs with sensors and actuators will mark the beginning of a new era where AI systems interact with the physical world, controlling everything from thermostats to industrial processes.The ease with which AI can now interact with humans has led to concerns about job displacement and the potential for AI to manipulate public opinion. However, the real challenge lies in ensuring that these systems are designed and used ethically, taking into account the potential impact on individuals and society. The trust deficit in AI is multifaceted, encompassing issues such as disinformation, safety and security, the enigma of the “black box,” ethical dilemmas, bias, instability, hallucinations in large language models, unforeseen risks, potential employment displacement and social disparities, environmental consequences, industry consolidation, and government intervention. To bridge this trust gap, it’s essential that we empower, train, and include humans in managing AI tools. This approach will not only ensure that AI systems are designed and used ethically but also provide a framework for addressing the complex challenges that arise from their integration into our daily lives. As we move forward, it’s crucial that we prioritise the development of AI systems that are not only powerful but also trustworthy and transparent. The path to achieving this lies in recognising the interconnectedness of all systems and the need for a collaborative approach to governance and economy. In 2024, we will take the first steps towards this future, and it’s essential that we do so with a deep understanding of the challenges and opportunities that lie ahead. The future of AI is not just about the technology; it’s about how we choose to use it to create a better world for all. Know more about the syllabus and placement record of our Top Ranked Data Science Course in Kolkata, Data Science course in Bangalore, Data Science course in Hyderabad, and Data Science course in Chennai.
The Forces Defining a Super Future
Here’s a whirlwind tour of the incredible developments that have unfolded at an unbelievable velocity giving rise to new economic value propositions and setting off geopolitical forces of techno-nationalism to reshape human civilisation In the blink of an eyelid, we are nearly a quarter of our way through this century that seems to have begun only a while ago. In this short span, the technological, scientific, social, environmental, and political changes that humanity has encountered have been breathtaking. Unprecedented global catastrophes like the Pandemic, climate change, and geopolitical tensions have often triggered these. Incredible technological advancements have unfolded the future at an unbelievable velocity giving rise to new economic value propositions and setting off political forces of techno-nationalism that are reshaping business, economics, our careers, and lives. The Answers to Critical Challenges Yet, despite all these remarkable events we still haven’t understood how photosynthesis happens; we haven’t fathomed the quantum world around us. How a leaf turns sunlight and water into food has vexed the most brilliant minds amongst us. Almost a year ago, futurist Michio Kaku, came out with his remarkable work, Quantum Supremacy. He unveiled a future where Quantum Computing would find answers to some of the most challenging problems of humankind such as a cure to cancer, or a limitless source of energy that helps us solve carbon emissions. In the first week of May this year, scientists at the MIT Plasma Science and Fusion Center reached what the university press described as a “major milestone” in the realm of fusion power plants. The breakthrough could “usher in an era of virtually limitless power production,” MIT News wrote. At the heart of this innovation was a new type of magnet created by the scientists. Made from a high-temperature superconducting material, it had a magnetic field strength of 20 tesla, which is a world record. (Just to set the perspective, a common refrigerator magnet is only around 0.001 tesla, while the incredibly strong magnets used in MRI machines are 3 tesla!) Technology That’s Paying Attention Since November 2022, humanity has been astonished by the emergence of Generative Artificial Intelligence that has democratised AI. It has been writing codes, essays, poems, songs, painting pictures and now even making movies that have made Hollywood stop in the tracks and in some cases halt investments in new studios. Nevertheless, such jaw-dropping human-like capabilities is nothing compared to recent developments when Claude3, a GenAI tool shocked experts with its apparent signs of awareness and self-actualisation. When engineers testing Claude 3 Opus asked the tool to pick out a target sentence hidden among a corpus of random documents – a task equivalent to finding a needle in a haystack for an AI. Not only did Opus find the so-called needle – it realised it was being tested. In fact, the model even expressed its suspicion that the sentence in question might have been “injected out of context” into documents only to test whether it was “paying attention”. AI Super Intelligence in 2 Years Commenting on AI’s ever-improving smarts, Musk said that if you think of an artificial general intelligence–an AI system that’s not specialised in one area of expertise – being compared to the smartest human, then he thinks that the moment where AIs and humans are as clever as each other will happen “probably next year, within two years.” Are we on the threshold of a Super Future? Does the “Super Future” represent a vision of the future that is highly advanced, technologically driven, and focused on sustainable and ethical progress, with significant implications for work, society, and the overall human experience? What are the forces that define this Super Future? These would span across technology, geopolitics, the future of work, and climate change. A Quantum Future The advancements of AI, Quantum Computing, space technology, etc driving the Fourth Industrial Revolution promise to unlock economic value that can double the global GDP. McKinsey analysis for the third annual Quantum Technology Monitor reveals that “four sectors–chemicals, life sciences, finance, and mobility–are likely to see the earliest impact from quantum computing and could gain up to $2 trillion by 2035” (www.mckinsey.com). In healthcare, quantum computing holds the promise of rapid drug development making possible new medicines within months of research by simulating complex chemical reactions and molecular interactions much more accurately and efficiently than classical computers. The convergence of healthcare and the metaverse holds the promise of amazing possibilities. The purpose of the metaverse, according to digital theorist Douglas Rushkoff, is to quantise and monetise more aspects of life, potentially leading to a shift away from market-driven virtual simulations towards reconnecting with the real world and nature. In the healthcare sector, the metaverse is anticipated to revolutionise healthcare by transcending traditional physical limitations, with healthcare professionals showing more enthusiasm for its adoption compared to patients. The metaverse’s impact on healthcare is expected to be positive, with potential applications such as virtual interactive healthcare human digital twins and virtual hospital experiences Quantum computers have the potential to break complex encryption algorithms, which could help banks and financial institutions better secure their sensitive data. Quantum computing could also enable the creation of more sophisticated financial models, leading to more accurate predictions and better investment decisions. These could be used to simulate complex chemical reactions, helping develop more efficient and cost-effective ways to produce and store energy. They could also aid in creating more accurate climate models to mitigate the effects of climate change. Techno-Nationalism & Trade Blocs Geopolitical tensions have been an unfortunate consequence of technological breakthroughs, as countries use techno nationalism to deny access to breakthroughs. Companies like Nvidia, Tesla, Apple, TikTok, Google, Huawei, ZTE, faced the brunt of such regulatory measures. The semiconductor war between China and the US has sharply divided the world. It is creating a new set of countries with technological supremacy, while others will struggle to catch up. Semiconductor wars were just a glimpse of a world that witnessed dramatic shifts in global trade patterns. In 2023, Mexico became
How Data Science is Paving the Way
The digital age has transformed data from mere operational by-products into a core strategic asset. Businesses across industries now recognise the latent power of data, often likening it to the new oil. However, as with crude oil, raw data’s value is only realised once it is refined and analysed. The Inherent Value of Data As Stephen DeAngelis, founder of Data Science firm Enterra Solutions, outlined in a recent blog post on LinkedIn, the question, “What’s the value of my data?” poses a significant challenge for many organisations. BillSchmarzo, Customer AI and Data Innovation Strategist at Dell Technologies and a leading voice in data analytics, emphasises the complexity of this question, suggesting that data’s value cannot be determined in isolation. Instead, its worth is tied to its ability to enhance business outcomes – whether through improving efficiency, reducing costs, optimising operations, or driving innovation. Many also believe data has usurped people as a company’s most valuable asset. The World Economic Forum’s classification of data as an asset class akin to oil underscores this shift, highlighting the necessity of refining data to extract its true value. This process involves leveraging data science and business analytics, disciplines that transform raw data into actionable insights. But there may often be a question surrounding this – how is data science any different from Business Analytics? The answer to this lies in their positioning in the company pipeline and their impact on business outcomes. Despite their differences, these disciplines are complementary. While data science involves using statistical techniques and machine learning to uncover patterns and predict outcomes from unstructured data, business analytics focuses on analysing structured data to inform specific business decisions. Data scientists often delve into exploratory analyses, tackling broad questions with no pre-defined answers. Business analysts, on the other hand, translate these findings into actionable business strategies. The intersection of these roles is where the true power of data is unleashed. Schmarzo outlines a framework for understanding data valuation that hinges on its application within business contexts. His approach emphasises several key principles: Contextual Integration: Data’s value emerges from its application to specific business and operational use cases. Incremental Analysis: A step-by-step, use-case-driven approach simplifies data valuation and amplifies its economic impact. Focused Utilisation: Organisations often fail not due to a lack of use cases but because they pursue too many, diluting their efforts. This pragmatic approach suggests that data’s value is dynamically linked to its contextual utility. Each incremental use case not only generates direct insights but also refines the data, enhancing its value for future applications. The Role of Embedded Analytics Embedded analytics integrates data analysis and visualisation directly into software applications, allowing users to analyse data within the application itself. This integration helps users identify issues and opportunities in real time without switching between different platforms, offering a seamless window into an application’s data. Often, this integration is white-labelled, meaning it is rebranded to match the host application’s look and feel, making the analytics appear as a native feature. Alternatively, it may be grey-labelled, retaining some branding from the analytics provider. By embedding rather than building analytics, developers can concentrate on enhancing the core product, relying on the analytics partner to provide continual improvements and new features, notes Yellowfin. For instance, Kodak’s Prinergy software incorporates embedded analytics, enabling users to monitor metrics like ink usage and production trends directly within the application. This functionality helps users forecast resource needs and enhance efficiency without the need for a separate analytics tool. It offers: Sustainable Competitive Advantage: Embedding modern analytics keeps an application ahead of competitors, providing actionable insights and meeting client expectations for integrated analytics. Enhanced Customer Experience: Modern analytics platforms offer advanced features like automated insights, interactive dashboards, and collaborative tools, significantly improving how customers interact with and benefit from your application. New Revenue Streams: Embedded analytics can be monetised through upsells, offering advanced features or additional capabilities as premium options. Faster Time to Market: Partnering with an established analytics provider speeds up the integration process, allowing quicker deployment and revenue generation. Historically, effective data analysis required the collaboration of domain experts, statisticians, and data engineers – a resource-intensive process. However, advancements in embedded analytics platforms have streamlined this workflow. Tools like the Enterra Autonomous Decision Science (ADS) platform integrate these capabilities, enabling business experts to leverage sophisticated analytics without extensive technical support. Henry Peter, Co-founder and CTO of Ushur, a customer experience automation firm, advocates for integrating business intelligence with operational workflows. He argues that actionable insights derived from machine learning should inform real-time business operations, driving immediate and strategic actions across the organisation. The Payoff Despite the challenges in data valuation, businesses are increasingly investing in advanced analytics. A study by IDC forecasts nearly 13% annual growth in global big data analytics spending until 2025. Several key factors drive this investment: Operational Risk Management: Analytics helps identify and mitigate risks such as fraud, data breaches, and operational failures. Performance Improvement: Data-driven insights enhance productivity and efficiency at both individual and organisational levels. Enhanced Marketing: It provides a granular understanding of customer behaviour, optimising marketing spend and improving ROI. Situational Awareness: Advanced analytics equips businesses to navigate dynamic environments such as in modelling numerous “what if” scenarios for strategic planning. The strategic value of data lies not in its raw form but in its refined, analysed state. Advanced analytics transforms data into a powerful tool for decision-making, risk management, and operational efficiency. As technology continues to evolve, the ability to seamlessly integrate data science and business analytics into everyday business operations will become a critical differentiator. Organisations that effectively leverage their data assets will enjoy reduced operational costs, enhanced decision-making capabilities, and accelerated growth. As Dell’s Schmarzo concludes, the continuous refinement and reuse of data and analytics not only generate immediate business value but also create a compounding effect, enhancing future applications and driving sustained economic benefits. Now is the time for businesses to unlock the potential of their data, turning it into a cornerstone of their strategic advantage.
AI in the Indian Job Market
Apart from technical knowhow, the biggest non-AI skills in demand for AI roles in India are communication, analytical skills and sales skills, says a recent report by LinkedIn. Gone are the days where coders just coded. Today, they must have sectoral understanding, contextualise problems, innovate and communicate well within their teams. That is the recipe for success. The rapid proliferation of Artificial Intelligence (AI) has been fundamentally reshaping industries and hiring across the globe. While companies with a ‘Head of AI’ position have more than tripled in the past five years itself, growing over 13% since the turn of 2023, the growth in the need for hires equipped with ‘people’ skills indicates that while professionals need technical skills, soft skills also need to be prioritised. The most recent LinkedIn Future of Work Report on AI wrote: “Balancing AI skills with people skills is critical to career growth. Tech professionals who have developed one or more of these ‘people’ skills — communication, teamwork, problem-solving, or leadership — in addition to hard skills get promoted more than 13% faster than employees who only have hard skills.” As AI technologies continue to evolve, they are catalysing significant shifts in the job market landscape, creating both challenges and opportunities for professionals across various sectors. Here we look at the sectors experiencing the most growth in AI-related jobs in India, the essential skills necessary to thrive in this environment, and their potential future implications. Sectoral Growth in AI-Related Jobs in India Several sectors in India have witnessed substantial growth in AI-related job opportunities: professional services, technology, information and media, financial services, administrative services and manufacturing rounding off the top 5 sectors, according to the report. The technology sector stands out as a frontrunner, with companies increasingly investing in AI-driven solutions to enhance productivity and innovation. Notably, software development, data analysis and machine learning engineering have emerged as key roles within this sector, offering lucrative career prospects for skilled professionals. Furthermore, the healthcare industry too has seen a surge in AI adoption, with applications ranging from disease diagnosis and treatment optimisation to personalised medicine. AI-powered medical imaging, predictive analytics, and virtual health assistants are revolutionising healthcare delivery, driving demand for professionals with expertise in AI algorithms, data science, and healthcare domain knowledge. The finance and banking sector is another significant player in the AI job market, leveraging machine learning algorithms for fraud detection, risk management, and algorithmic trading. As financial institutions embrace digital transformation, the demand for AI specialists, quantitative analysts, and cybersecurity experts continues to rise, creating diverse opportunities for individuals with a blend of technical and financial acumen. Moreover, e-commerce and retail companies are also increasingly harnessing AI technologies to personalise customer experiences, optimise supply chain operations, and drive sales growth. Roles such as e-commerce data analyst, AI product manager, and digital marketing strategist are in high demand, reflecting the industry’s shift towards data-driven decision-making and AI-driven innovation. Skills for Success In light of evolving labour market dynamics, acquiring and honing relevant skills is essential for professionals seeking to thrive in AI-related roles. In India, knowledge of data structures, Machine Learning (ML) and Natural Language Processing (NLP) were the three most in-demand AI skills in job postings on LinkedIn since December 2022. Source: LinkedIn Technical proficiency in programming languages such as Python, R, and Java is of course paramount, as AI applications heavily rely on software development and algorithm implementation. Among the 41,000-odd distinct standardised skills coded and classified by LinkedIn taxonomists, the top skills in the AI category include machine learning, natural language processing, data structures, AI, computer vision, image processing, deep learning, TensorFlow, Pandas (software), and OpenCV, among others. Additionally, expertise in machine learning frameworks like PyTorch and scikit-learn is highly valued, enabling individuals to build, train, and deploy AI models effectively. Furthermore, a solid foundation in mathematics and statistics is indispensable for understanding the theoretical underpinnings of machine learning algorithms and data analysis techniques. Proficiency in linear algebra, calculus, and probability theory equips professionals with the necessary mathematical tools to tackle complex AI problems and derive meaningful insights from data. Domain-specific knowledge is also becoming increasingly valuable in AI-related roles, as it enables professionals to contextualise AI solutions within specific industries and domains. Whether in healthcare, finance, e-commerce, or other sectors, understanding the nuances of the industry landscape, regulatory frameworks, and business challenges is crucial for developing tailored AI solutions that deliver tangible value. Perhaps most crucially, soft skills such as critical thinking, problem-solving, and effective communication are now being regarded as indispensable for success in AI-related roles. The ability to collaborate with multidisciplinary teams, articulate technical concepts to non-technical stakeholders, and adapt to evolving challenges is essential in today’s dynamic job market. The LinkedIn report cited communication skills, analytical skills and sales skills as the three biggest non-AI skills in demand in AI roles in India. Future Implications Looking ahead, the proliferation of AI technologies is poised to have far-reaching implications for the job market in India. While AI-driven automation may disrupt certain traditional roles, it also presents opportunities for job creation and skill upgradation across various sectors. As repetitive tasks become automated, there will be a greater emphasis on roles that require human creativity, empathy, and strategic decision-making. Moreover, the democratisation of AI tools and resources is lowering barriers to entry, enabling individuals from diverse backgrounds to pursue careers in AI-related fields. Online courses, MOOCs, and self-paced learning platforms offer accessible avenues for acquiring technical skills and staying abreast of industry trends, empowering individuals to adapt to the changing job market landscape. Furthermore, interdisciplinary collaboration is expected to become increasingly prevalent, as AI intersects with other emerging technologies such as blockchain, Internet of Things (IoT), and augmented reality (AR). Professionals with hybrid skill sets spanning multiple domains will be well-positioned to capitalise on these converging trends and drive innovation across industries. The Indian AI labour market is undergoing a profound transformation driven by the rise of AI technologies. While certain sectors are experiencing rapid growth in AI-related jobs,
The Unseen Threads of Chaos: Predicting Black Swan Events with AI and Quantum Computing
While traditional tools often fall short, artificial intelligence and quantum computing promise to turn chaos into insight by mining unconventional data sources and modelling the intricate interconnections of our world It took just 11 tumultuous days for the Bashar al-Assad regime to collapse in the face of an attack by a rag-tag bunch of armed militias, ending the half-a-century autocratic reign of the Assad family had that started with his father Hafeez al-Assad assuming power over 50 years ago. The event sent shockwaves through geopolitical landscapes, blindsiding analysts and citizens alike. In 2011 the earthquake and tsunami in Japan disrupted the supply chains of various industries, particularly electronics and automotive, due to the concentration of manufacturing facilities in the affected region. This event exposed the risks associated with geographic concentration of production.These were classic Black Swan events – rare, impactful, and utterly unforeseen by most. But could these have been foreseen? Could the ripples that preceded the storm have been detected, hidden as they were in a chaotic sea of political, economic, and social signals? This is the crux of Black Swan analysis: the attempt to predict the unpredictable. While traditional tools often fall short, artificial intelligence (AI) and quantum computing may offer new hope. By mining unconventional data sources and modelling the intricate interconnections of our world, these technologies promise to turn chaos into insight. What is a Black Swan Event? The idea of Black Swan events, coined by scholar Nassim Nicholas Taleb, is simple yet profound. These are events that are extremely rare, have outsized impacts, and are only explainable in hindsight. They’re like a sudden avalanche in an otherwise tranquil mountain range – seemingly spontaneous but often rooted in subtle warning signs that went unnoticed. The collapse of Assad’s regime serves as a prime example. Years of political tension, economic strain, and shifting alliances created a delicate balance that ultimately tipped in unexpected ways. Yet the signs – the proverbial “snowflakes” that triggered the avalanche – were scattered across countless domains, from local dissent to international policy. The Power of Unconventional Data Predicting Black Swans is not about looking for single, obvious causes; it’s about identifying faint signals buried in noise. Imagine trying to forecast a thunderstorm by observing not just the clouds overhead but also the moisture content of the soil, the flight patterns of birds, and even the number of people complaining about the humidity on social media. Companies like RS Metrics provide satellite imagery of store parking lots, allowing analysts to gauge foot traffic at big-box retailers (such as Walmart or Home Depot) well ahead of quarterly sales announcements. More cars in the lot often signal higher in-store customer activity, which can predict stronger sales. Individually, these signals might seem trivial, but together, they could reveal the storm’s approach. This is where alternative data comes in. Unlike traditional datasets like GDP figures or stock prices, alternative data sources draw from unconventional areas such as: Social Media Trends: Sentiment analysis on platforms like Twitter can reveal shifts in public mood or collective anxiety, offering early warnings of political unrest or market instability. Satellite Imagery: Monitoring nighttime light intensity in urban areas can track economic activity or detect unusual patterns like power outages in conflict zones. Climate Data: Understanding localised weather anomalies or deforestation rates can uncover risks that might cascade into food shortages or migration crises. Supply Chain Activity: AI-powered tools can track shipping and logistics patterns to flag disruptions in trade before they escalate. Each of these data sources acts like a piece of a puzzle. Individually, they may not make sense, but when woven together by AI, they can paint a clearer picture of emerging risks. How AI is Redefining Black Swan Analysis If traditional analysis is like fishing with a net, AI is akin to using a radar to map the entire ocean. By analysing vast and disparate datasets, AI can spot patterns and correlations that humans might miss. Here’s how: Deep Pattern Recognition: AI algorithms, particularly neural networks, excel at finding hidden relationships in data. For instance, a spike in social media posts about inflation might correlate with subtle changes in consumer spending patterns, hinting at an economic tipping point. Dynamic Simulations: AI can simulate countless scenarios to explore how small disruptions might cascade through a system. Think of it as predicting how a single pebble might trigger an avalanche, tracing its path through every possible crevice. Cross-Disciplinary Insights: By integrating data from diverse fields – economics, ecology, politics – AI provides a holistic view of risk. For example, it might reveal how droughts in one region could disrupt global food prices, fuelling social unrest elsewhere. The Role of Quantum Computing: Mapping the Unmappable While AI is already transforming Black Swan prediction, quantum computing takes this potential to another level. Traditional computers struggle to model highly complex systems because they process one calculation at a time. Quantum computers, however, can process multiple possibilities simultaneously, making them uniquely suited for tackling problems with vast interdependencies. For example, predicting the collapse of a political regime like Assad’s requires analysing countless variables: economic sanctions, shifting alliances, public sentiment, and even military dynamics. Quantum computers can model these variables in parallel, exploring millions of scenarios in the time it takes traditional computers to analyse one. This allows us to map “what if” scenarios with unprecedented speed and accuracy. A New Era of Preparedness AI and quantum computing won’t eliminate Black Swan events entirely – after all, by definition, they’re unpredictable. But they can help us see the fault lines before they rupture, turning the unknowable into something actionable. Imagine if early warning signs of Assad’s collapse had been identified through AI analysis of satellite images showing mass troop movements, combined with sentiment analysis from social media posts in Syria’s neighbouring regions. Or if a quantum computer had simulated scenarios where economic sanctions aligned with internal dissent to destabilise the regime. Such insights could have informed better policymaking or humanitarian responses, mitigating the fallout. The
Untangling the Data Mesh
Data mesh decentralises ownership to enable a self-serve infrastructure, represents a groundbreaking shift in how organisations approach data management and directly addresses the limitations of traditional centralised models. As industries continue to generate and rely on vast amounts of data, innovative frameworks like data mesh will be critical for unlocking the full potential of data-driven decision-making. In the 2010s, companies like Netflix and Uber faced a monumental challenge: their existing centralised data systems couldn’t scale to meet the demands of rapidly expanding user bases and increasingly complex analytics needs. Data lakes and warehouses became bottlenecks – plagued by delays, inconsistent data quality and inefficiencies caused by overburdened centralised teams. Recognising these limitations, ZhamakDehghani, then a technologist at ThoughtWorks, introduced the concept of a ‘data mesh’. Her vision was to decentralise data ownership, empowering domain-specific teams to manage and serve their own data as products. This architecture addressed the growing need for scalability, domain autonomy and enhanced data quality, laying the groundwork for modern data innovation. What is Data Mesh Architecture? Data mesh is a decentralised approach to data management that prioritises domain-oriented design. Unlike centralised data lakes, where all organisational data is aggregated into a single repository, data mesh distributes data ownership across teams, each responsible for specific datasets or “data products.” This shift fosters better data quality, improved scalability and enhanced decision-making capabilities. The Core Principles of Data Mesh Domain-Oriented Decentralised Data Ownership: Data is managed by the teams that generate and use it. This domain-focused ownership ensures that the data remains relevant, accurate and actionable. Data as a Product: Teams treat their datasets as products, emphasising user-centric design, quality assurance and accessibility. The goal is to create datasets that meet the needs of internal and external stakeholders. Self-Serve Data Infrastructure: A self-serve infrastructure provides domain teams with the tools and platforms needed to manage their data independently, reducing reliance on centralised IT teams. Federated Computational Governance: Governance is implemented at a federated level, ensuring compliance, security and consistency across decentralised datasets without stifling team autonomy. For a data mesh to function effectively, clear roles must be defined for the individuals performing tasks within the system. Ownership is assigned to team archetypes or functions, each managing core user journeys. These roles, which can be adapted to suit the enterprise’s needs, are not always directly tied to specific employees or teams. A data domain is typically aligned with a business unit (BU) within the organisation, such as HR, finance, or marketing with two primary domain functions: data producer teams and data consumer teams. A single data domain can serve both functions, where the producer team creates data products, and the consumer team uses these products for insights or to generate new data products. In addition to domain-specific teams, centralised functions oversee cross-domain governance and services. These teams help manage the operational burden, ensuring compliance and facilitating inter-domain interactions essential for the mesh’s success. Key roles within the data mesh include: Data Domain Producer Teams: They build and maintain data products throughout their lifecycle. Data Domain Consumer Teams: They discover and utilise data products for analysis or other purposes. Central Data Governance Team: They define and enforce policies, ensuring data quality and trustworthiness. Central Self-Service Data Infrastructure Team: This team provides the infrastructure and tooling for data producers and consumers. Why use it? The main pros of the architecture include: Scalability: Decentralised management allows organisations to handle growing data volumes more effectively. Improved Data Quality: Domain experts maintain their datasets, ensuring relevance and accuracy. Agility: Teams can respond quickly to changing business needs without waiting for centralised teams to process data. Enhanced Collaboration: Cross-domain collaboration is facilitated by standardised governance and infrastructure. While benefits of a data mesh are significant, successful implementation requires overcoming several challenges. The most critical of these is cultivating a culture where teams take ownership of their data and treat datasets as products. Once this foundation is established, investing in the right tools and training is crucial for building a self-serve infrastructure – a step that many organisations hesitate to take. In implementation, achieving a balance between autonomy and compliance is essential. This balance requires careful planning and execution, ultimately determining the success or failure of the model. Advancements in Data Mesh Recent technological advancements have bolstered the adoption and effectiveness of data mesh architectures: Cloud-Native Platforms: Cloud providers such as AWS and Google Cloud now offer robust support for data mesh, enabling seamless integration of domain-specific data products across distributed environments. Data Observability Tools: These tools enhance monitoring and management of data quality and performance, ensuring reliability in a decentralised setup. AI-Powered Analytics: Artificial intelligence and machine learning tools enable domain teams to derive insights more efficiently, further emphasising the value of treating data as a product. API-Driven Connectivity: APIs simplify data sharing and interoperability across domains, facilitating real-time collaboration and integration. Vertical Use Cases Data mesh is rapidly gaining traction across verticals, demonstrating high versatility: Financial Services: In banking and investment sectors, data mesh enables departments to manage and analyse their specific datasets, such as customer behaviour, compliance and market trends. This decentralisation enhances agility and reduces bottlenecks, improving customer experiences and regulatory compliance. E-Commerce: Online retailers manage diverse datasets, including sales, inventory and customer preferences. Data mesh empowers domain teams to optimise operations, implement personalised marketing strategies and streamline supply chains. Healthcare: Hospitals and research institutions use data mesh to integrate patient records, clinical trial results and administrative data. This approach improves patient outcomes by enabling tailored treatments and reducing data silos. Manufacturing: Using data mesh, manufacturers can optimise production processes, enhance supply chain transparency and predict equipment maintenance needs through domain-specific analytics. Government Agencies: Public sector organisations leverage data mesh for enhanced data sharing and collaboration between departments, enabling better policy-making and public services.