Edge Computing in the Age of AI: An Overview
Wed, 27 Sep 2023 05:19:01 -0000
|Read Time: 0 minutes
Introduction to Edge Computing
Edge computing is a distributed computing paradigm that brings data processing and analysis closer to the source of data generation, rather than relying on centralized cloud or datacenter resources. It aims to address the limitations of traditional cloud computing, such as latency, bandwidth constraints, privacy concerns, and the need for real-time decision-making.
Edge Computing Use Cases
Edge computing finds applications across various industries, including manufacturing, transportation, healthcare, retail, agriculture, and digital cities. It empowers real-time monitoring, control, and optimization of processes. This enables efficient data analysis and decision-making at the edge as it complements cloud computing by providing a distributed computing infrastructure.
Here are some common examples:
- Industrial internet of things (IIoT)—Edge computing enables real-time monitoring, control, and optimization of industrial processes. It can be used for predictive maintenance, quality control, energy management, and overall operational efficiency improvements.
- Digital cities—Edge computing supports the development of intelligent and connected urban environments. It can be utilized for traffic management, smart lighting, waste management, public safety, and environmental monitoring.
- Autonomous vehicles—Edge computing plays a vital role in autonomous vehicle technology. By processing sensor data locally, edge computing enables real-time decision-making, reducing reliance on cloud connectivity and ensuring quick response times for safe navigation.
- Healthcare—Edge computing helps in remote patient monitoring, telemedicine, and real-time health data analysis. It enables faster diagnosis, personalized treatment, and improved patient outcomes.
- Retail—Edge computing is used in retail for inventory management, personalized marketing, loss prevention, and in-store analytics. It enables real-time data processing for optimizing supply chains, improving customer experiences, and implementing dynamic pricing strategies.
- Energy management—Edge computing can be employed in smart grids to monitor energy consumption, optimize distribution, and detect anomalies. It enables efficient energy management, load balancing, and integration of renewable energy sources.
- Surveillance and security—Edge computing enhances video surveillance systems by enabling local video analysis, object recognition, and real-time threat detection. It reduces bandwidth requirements and enables faster response times for security incidents.
- Agriculture—Edge computing is utilized in precision farming for monitoring and optimizing crop conditions. It enables the analysis of sensor data related to soil moisture, weather conditions, and crop health, allowing farmers to make informed decisions regarding irrigation, fertilization, and pest control.
These are just a few examples, and the applications of edge computing continue to expand as technology advances. The key idea is to process data closer to its source, reducing latency, improving reliability, and enabling real-time decision-making for time-sensitive applications.
The Challenges with Edge Computing
Edge computing brings numerous benefits, but it also presents a set of challenges that organizations need to address. The following image highlights some common challenges associated with edge computing:
Edge Computing Architecture Overview
The following diagram represents a typical edge computing architecture and its associated taxonomy.
A typical edge computing architecture consists of several components working together to enable data processing and analysis at the edge. Here are the key elements you would find in such an architecture:
- Edge devices—These are the devices deployed at the network edge, such as sensors, IoT devices, gateways, or edge servers. They collect and generate data from various sources and act as the first point of data processing.
- Edge gateway—An edge gateway is a device that acts as an intermediary between edge devices and the rest of the architecture. It aggregates and filters data from multiple devices, performs initial pre-processing, and ensures secure communication with other components.
- Edge computing infrastructure—This includes edge servers or edge nodes deployed at the edge locations. These servers have computational power, storage, and networking capabilities. They are responsible for running edge applications and processing data locally.
- Edge software stack—The edge software stack consists of various software components installed on edge devices and servers. It typically includes operating systems, containerization technologies (such as Docker or Kubernetes), and edge computing frameworks for deploying and managing edge applications.
- Edge analytics and AI—Edge analytics involves running data analysis and machine learning algorithms at the edge. This enables real-time insights and decision-making without relying on a centralized cloud infrastructure. Edge AI refers to the deployment of artificial intelligence algorithms and models at the edge for local inference and decision-making. The next section: Edge Inferencing describes the main use case in this regard.
- Connectivity—Edge computing architectures rely on connectivity technologies to transfer data between edge devices, edge servers, and other components. This can include wired and wireless networks, such as Ethernet, Wi-Fi, cellular networks, or even specialized protocols for IoT devices.
- Cloud or centralized infrastructure—While edge computing emphasizes local processing, there is often a connection to a centralized cloud or data center for certain tasks. This connection allows for remote management, data storage, more resource-intensive processing, or long-term analytics. Those resources are often broken down into two tiers – near and far edge:
- Far edge: Far edge refers to computing infrastructure and resources that are located close to the edge devices or sensors generating the data. It involves placing computational power and storage capabilities in proximity to where the data is produced. Far edge computing enables real-time or low-latency processing of data, reducing the need for transmitting all the data to a centralized cloud or datacenter.
- Near edge: Near edge, sometimes referred to as the "cloud edge" or "remote edge" describes computing infrastructure and resources that are positioned farther away from the edge devices. In the near edge model, data is typically collected and pre-processed at the edge, and then transmitted to a more centralized location, such as a cloud or datacenter for further analysis, storage, or long-term processing.
- Management and orchestration—To effectively manage the edge computing infrastructure, there is a need for centralized management and orchestration tools. These tools handle tasks like provisioning, monitoring, configuration management, software updates, and security management for the edge devices and servers.
It is important to note that while the components and the configurations of edge solution may differ, the overall objective remains the same: to process and analyze data at the edge to achieve real-time insights, reduced latency, improved efficiency, and better overall performance.
Edge Inferencing
Data growth driven by data intensive applications and ubiquitous sensors to enable real time insight is growing three times faster than access network. This drives data processing at the edge to keep up with the pace and reduce cloud cost and latency. IDC estimates that by 2027 62% of enterprises data will be processed at the edge!
Inference at the edge is a technique that enables data-gathering from devices to provide actionable intelligence using AI techniques rather than relying solely on cloud-based servers or data centers. It involves installing an edge server with an integrated AI accelerator (or a dedicated AI gateway device) close to the source of data, which results in much faster response time.1 This technique improves performance by reducing the time from input data to inference insight, and reduces the dependency on network connectivity, ultimately improving the business bottom line.2 Inference at the edge also improves security as the large dataset do not have to be transferred to the cloud.3
Final Notes
Edge computing in the age of AI marks a significant paradigm shift in how data is processed, and insights are generated. By bringing AI to the edge, we can unlock real-time decision-making, improve efficiency, and enable innovations across various industries. While challenges exist, advancements in hardware, software, and security are paving the way for a future where intelligent edge devices are an integral part of our interconnected world.
It is expected that Inferencing market alone will overtake training with highest growth at the edge – necessitating competition in data center, near edge, and far edge.
For more information on how edge-inferencing works, refer to the next post on this regard: Inferencing at the Edge
1 https://steatite-embedded.co.uk/what-is-ai-inference-at-the-edge/
2 https://www.storagereview.com/review/edge-inferencing-is-getting-serious-thanks-to-new-hardware
Related Blog Posts
DevEdgeOps Defined
Mon, 25 Sep 2023 07:30:48 -0000
|Read Time: 0 minutes
What is a DevEdgeOps Platform?
As the demand for edge computing continues to grow, organizations are seeking comprehensive solutions that streamline the development and operational processes in edge environments. This has led to the emergence of DevEdgeOps platforms, with specialized processes and tools. Including frameworks designed to support the unique requirements of developing, deploying, and managing applications in edge computing architectures.
Edge Operations Shift Left
Shift Left refers to the practice of moving activities that were traditionally performed mostly at production stage to an earlier in the development stage. It is often applied in software development and DevOps to integrate testing, security, and other considerations earlier in the development lifecycle. Similarly, in the world of edge computing, we are moving operational tasks to an earlier stage, just like how we did with Shift Left in DevOps. We call this new idea, DevEdgeOps.
A DevEdgeOps platform facilitates collaboration between developers and operations teams, addressing challenges like network connectivity, security, scalability, and edge deployment management.
In this blog post, we introduce edge computing, its use cases, and architecture. We explore DevEdgeOps platforms, discussing their features and impact on edge computing development and operations.
Introduction to Edge Computing
Edge computing is a distributed computing paradigm that brings data processing and analysis closer to the source of data generation, rather than relying on centralized cloud or datacenter resources. It aims to address the limitations of traditional cloud computing, such as latency, bandwidth constraints, privacy concerns, and the need for real-time decision-making.
To learn more about the use cases, unique challenges, typical architecture and taxonomy refer to my previous post: Edge Computing in the Age of AI: An Overview
DevEdgeOps
DevEdgeOps is a term that combines elements of development (DevOps) and edge computing. It refers to the practices, methodologies, and tools used for managing and deploying applications in edge computing environments while leveraging the principles of DevOps. In other words, it aims to enable efficient development, deployment, and management of applications in edge computing environments, combining the agility and automation of DevOps with the unique requirements of edge deployments.
DevEdgeOps Platform
A DevEdgeOps platform provides developers and operations teams with a unified environment for managing the entire lifecycle of edge applications, from development and testing to deployment and monitoring. These platforms typically combine essential DevOps practices with features specific to edge computing, allowing organizations to build, deploy, and manage edge applications efficiently.
Key Features of DevEdgeOps Platforms
- Centralized edge application management—DevEdgeOps platforms provide centralized management capabilities for edge applications. They offer dashboards, interfaces, and APIs that allow operations teams to monitor the health, performance, and status of edge deployments in real-time. These platforms may also include features for configuration management, remote troubleshooting, and log analysis, enabling efficient management of distributed edge nodes.
- Integration with edge infrastructure—DevEdgeOps platforms often integrate with edge infrastructure components such as edge gateways, edge servers, or cloud-based edge computing services. This integration simplifies the deployment process by providing seamless connectivity between the development platform and the edge environment, facilitating the deployment and scaling of edge applications.
- Edge-aware development tools—DevEdgeOps platforms offer development tools tailored for edge computing. These tools assist developers in optimizing their applications for edge environments, providing features such as code editors, debuggers, simulators, and testing frameworks specifically designed for edge scenarios.
- CI/CD pipelines for edge deployments—DevEdgeOps platforms enable the automation of continuous integration and deployment processes for edge applications. They provide pre-configured pipelines and templates that consider the unique requirements of edge environments, including packaging applications for different edge devices, managing software updates, and orchestrating deployments to distributed edge nodes.
- Edge simulation and testing capabilities—DevEdgeOps platforms often include simulation and testing features that help developers validate the functionality and performance of edge applications in various scenarios. These features simulate edge-specific conditions such as low-bandwidth networks, intermittent connectivity, and edge device failures, allowing developers to identify and address potential issues proactively.
Final Words
The emergence of new edge use cases that combine cloud-native infrastructure and AI introduces an increased operational complexity and demands more advanced application lifecycle management. Traditional management approaches may no longer be sustainable or efficient in addressing these challenges.
In my previous post, How the Edge Breaks DevOps, I referred to the unique challenges that the edge introduces and the need for a generic platform that will abstract the complexity associated with those. In this blog, I introduced DevEdgeOps platforms that combine essential DevOps practices with features specific to edge computing. I also described the set of features that are expected to be part of this new platform category. By embracing these approaches, organizations can effectively manage operational complexity and fully harness the potential of edge computing and AI.
Will AI Replace Software Developers?
Wed, 01 May 2024 10:41:32 -0000
|Read Time: 0 minutes
Over the past year, I have been actively involved in generative artificial intelligence (Gen AI) projects aimed at assisting developers in generating high-quality code. Our team has also adopted Copilot as part of our development environment. These tools offer a wide range of capabilities that can significantly reduce development time. From automatically generating commit comments and code descriptions to suggesting the next logical code block, they have become indispensable in our workflow.
According to a recent study by McKinsey, quantify the level of productivity gain in the following areas:
Figure 1. Software engineering: speeding developer work as a coding assistant (McKinsey)
This study shows that “The direct impact of AI on the productivity of software engineering could range from 20 to 45 percent of current annual spending on the function. This value would arise primarily from reducing time spent on certain activities, such as generating initial code drafts, code correction and refactoring, root-cause analysis, and generating new system designs. By accelerating the coding process, Generative AI could push the skill sets and capabilities needed in software engineering toward code and architecture design. One study found that software developers using Microsoft’s GitHub Copilot completed tasks 56 percent faster than those not using the tool. An internal McKinsey empirical study of software engineering teams found those who were trained to use generative AI tools rapidly reduced the time needed to generate and refactor code and engineers also reported a better work experience, citing improvements in happiness, flow, and fulfilment.”
What Makes the Code Assistant (Copilot) the Killer App for Gen AI?
The remarkable progress of AI-based code generation owes its success to the unique characteristics of programming languages. Unlike natural language text, code adheres to a structured syntax with well-defined rules. This structure enables AI models to excel in analyzing and generating code.
Several factors contribute to the swift evolution of AI-driven code generation:
- Structured nature of code–Code follows a strict format, making it amenable to automated analysis. The consistent structure allows AI algorithms to learn patterns and generate syntactically correct code.
- Validation tools–Compilers and other development tools play a crucial role. They validate code for correctness, ensuring that generated code adheres to language specifications. This continuous feedback loop enables AI systems to improve without human intervention.
- Repeatable work identification–AI excels at identifying repetitive tasks. In software development, there are numerous areas where routine work occurs, such as boilerplate code, data transformations, and error handling. AI can efficiently recognize and automate these repetitive patterns.
From Coding Assistant to Fully-Autonomous AI Software Engineer
The Cognition & Development Lab at Washington University in St. Louis investigates how infants and young children think, reason, and learn about the world around them. Their research focuses on the development of early social-cognitive capacities. They are the makers of Devin, the world’s first AI software engineer.
Devin possesses remarkable capabilities in software development in the following areas:
- Complex engineering tasks–With advances in long-term reasoning and planning, Devin can plan and execute complex engineering tasks that involve thousands of decisions. Devin recalls relevant context at every step, learns over time, and even corrects mistakes.
- Coding and debugging–Devin can write code, debug, and address bugs in codebases. It autonomously finds and fixes issues, making it a valuable teammate for developers.
- End-to-end app development–Devin builds and deploys apps from scratch. For example, it can create an interactive website, incrementally adding features requested by the user and deploying the app.
- AI model training and fine-tuning–Devin sets up fine-tuning for large language models, demonstrating its ability to train and improve its own AI models.
- Collaboration and communication–Devin actively collaborates with users. It reports progress in real-time, accepts feedback, and engages in design choices as needed.
- Real-world challenges–Devin tackles real-world GitHub issues found in open-source projects. It can also contribute to mature production repositories and address feature requests. Devin even takes on real jobs on platforms like Upwork, writing and debugging code for computer vision models.
The Devin project is a clear indication of how fast we move from simple coding assistants to more complete engineering capabilities.
Will AI Replace Software Developers?
When I asked this question recently during a Copilot training session that our team took, the answer was “No”, or to be more precise “Not yet”. The common thinking is that it provides a productivity enhancement tool that will save developers from spending time on tedious tasks such as documentation, testing, and so on. This could have been true yesterday, but as seen with project Devin, it already goes beyond simple assistance to full development engineering. We can rely on the experience from past transformations to learn a bit more about where this is all heading.
Learning from Cloud Transformation: Parallels with Gen AI Transformation
The advent of cloud computing, pioneered by AWS approximately 15 years ago, revolutionized the entire IT landscape. It introduced the concept of fully automated, API-driven data centers, significantly reducing the need for traditional system administrators and IT operations personnel. However, beyond the mere shrinking of the IT job market, the following parallel events unfolded:
- Traditional IT jobs shrank significantly–Small to medium-sized companies can now operate their IT infrastructure without dedicated IT operators. The cloud’s self-service capabilities have made routine maintenance and management more accessible.
- Emergence of new job titles: DevOps, SRO, and more–As organizations embrace cloud technologies, new roles emerge. DevOps engineers, site reliability operators (SROs), and other specialized positions became essential for optimizing cloud-based systems.
- The rise of SaaS startups–Cloud computing lowered the barriers of entry for delivering enterprise-grade solutions. Startups capitalized on this by becoming more agile and growing faster than established incumbents.
- Big tech companies’ accelerated growth–Tech giants like Google, Facebook, and Microsoft swiftly adopted cloud infrastructure. The self-service nature of APIs and SaaS offerings allowed them to scale rapidly, resulting in record growth rates.
Impact on Jobs and Budgets
While traditional IT jobs declined, the transformation also yielded positive outcomes:
- Increased efficiency and quality–Companies produced more products of higher quality at a fraction of the cost. The cloud’s scalability and automation played a pivotal role in achieving this.
- Budget shift from traditional IT to cloud–Gartner’s IT spending reports reveal a clear shift in budget allocation. Cloud investments have grown steadily, even amidst the disruption caused by the introduction of cloud infrastructure, see the following figure:
Figure 2. Cloud transformation’s impact on IT budget allocation
Looking Ahead: AI Transformation
As we transition to the era of AI, we can anticipate similar trends:
- Decline in traditional jobs–Just as cloud computing transformed the job landscape, AI adoption may lead to the decline of certain traditional roles.
- Creation of new jobs–Simultaneously, AI will create novel opportunities. Roles related to AI development, machine learning, and data science will flourish.
Short Term Opportunity
Organizations will allocate more resources to AI initiatives. The transition to AI is not merely an evolutionary step; it is a strategic imperative.
According to a research conducted by ISG on behalf of Glean, Generative AI projects consumed an average of 1.5 percent of IT budgets in 2023. These budgets are expected to rise to 2.7 percent in 2024 and further increase to 4.3 percent in 2025. Organizations recognize the potential of AI to enhance operational efficiency and bridge IT talent gaps. Gartner predicts that Generative AI impacts will be more pronounced in 2025. Despite this, worldwide IT spending is projected to grow by 8 percent in 2024. Organizations continue to invest in AI and automation to drive efficiency. The White House budget proposes allocating $75 billion for IT spending at civilian agencies in 2025. This substantial investment aims to deliver simple, seamless, and secure government services through technology.
The impact of AI extends far beyond the confines of the IT job market. It permeates nearly every facet of our professional landscape. As with any significant transformation, AI presents both risks and opportunities. Those who swiftly embrace it are more likely to seize the advantages.
So, what steps can software developers take to capitalize on this opportunity?
Tips for Software Developers in the Age of AI
In the immediate term, developers can enhance their effectiveness when working with AI assistants by acquiring a combination of the following technical skills:
- Learn AI basics–I would recommend starting the learning with AI Terms 101. I also recommend following the leading AI podcasts. I found this useful to keep myself up to date in this space and learn some useful tips and updates from industry experts.
- Use coding assistant tools (Copilot)–Coding assistant tools are definitely the low-hanging fruit and probably the simplest step to get into the AI development world. There is a growing list of tools that are available and can be integrated seamlessly into your existing development IDE. The following provides a useful reference to The Top 11 AI Coding Assistants to Use in 2024.
- Learn machine learning (ML) and deep learning concepts–Understanding the fundamentals of ML and deep learning is crucial. Familiarize yourself with neural networks, training models, and optimization techniques.
- Data science and analytics–Developers should grasp data preprocessing, feature engineering, and model evaluation. Proficiency in tools like Pandas, NumPy, and scikit-learn is beneficial.
- Frameworks and tools–Learn about popular AI frameworks such as TensorFlow, and PyTorch. These tools facilitate model building and deployment.
More skilled developers will need to learn how to create their own “AI engineers” which they will train and fine tune to assist them with user interface (UI), backend, and testing development tasks. They could even run a team of “AI engineers” to write an entire project.
Will AI Reduce the Demand for Software Engineers?
Not necessarily. In the case of cloud transformation, developers with AI expertise will likely be in high demand. Those who will not be able to adapt to this new world are likely to stay behind and face the risk of losing their job.
It would be fair to assume that the scope of work, post-AI transformation, will grow and will not stay stagnant. As an example, we will likely see products adding more “self-driving” capabilities, where they could run more complete tasks without the need for human feedback or enable close to human interaction with the product.
Under this assumption, the scope of new AI projects and products is going to grow, and that growth should balance the declining demand for traditional software engineering jobs.
Conclusion
As a history enthusiast, I often find parallels in the past that can serve as a guide to our future. The industrial era witnessed disruptive technological advancements that reshaped job markets. Some professions became obsolete, while new ones emerged. As a society, we adapted quickly, discovering new growth avenues. However, the emergence of AI presents unique challenges. Unlike previous disruptions, AI simultaneously impacts a wide range of job markets and progresses at an unparalleled pace. The implications are indeed profound.
Recent research by Nexford University on How Will Artificial Intelligence Affect Jobs 2024-2030 reveals some startling predictions. According to a report by the investment bank Goldman Sachs, AI could potentially replace the equivalent of 300 million full-time jobs. It could automate a quarter of the work tasks in the US and Europe, leading to new job creation and a productivity surge. The report also suggests that AI could increase the total annual value of goods and services produced globally by 7 percent. It predicts that two-thirds of jobs in the US and Europe are susceptible to some degree of AI automation, and around a quarter of all jobs could be entirely performed by AI.
The concerns raised by Yuval Noa Harari, a historian and professor at the Department of History of the Hebrew University of Jerusalem, resonate with many. The rapid evolution of AI may indeed lead to significant unemployment.
However, when it comes to software engineers, we can assert with confidence that regardless of how automated our processes become, there will always be a fundamental need for human expertise. These skilled professionals perform critical tasks such as maintenance, updates, improvements, error corrections, and the setup of complex software and hardware systems. These systems often require coordination among multiple specialists for optimal functionality.
In addition to these responsibilities, computer system analysts play a pivotal role. They review system capabilities, manage workflows, schedule improvements, and drive automation. This profession has seen a surge in demand in recent years and is likely to remain in high demand.
In conclusion, AI represents both risk and opportunity. While it automates routine tasks, it also paves the way for innovation. Our response will ultimately determine its impact.
References
- Economic potential of generative AI | McKinsey
- Introducing Devin, the first AI software engineer (cognition-labs.com)
- IT Spending & Budgets: Trends & Forecasts 2024
- Organizations continue to invest in AI and automation to drive efficiency
- This substantial investment aims to deliver simple, seamless, and secure government services through technology
- AI Terms 101: An A to Z AI Terminology Guide for Beginners
- 11 AI Podcasts That Will Shape Your Perspective (geekflare.com)\
- How Will Artificial Intelligence Affect Jobs 2024-2030 | Nexford University
- The Top 11 AI Coding Assistants to Use in 2024 | DataCamp
- Yuval Harari On The Future of Jobs & Technology, Intelligence vs Consciousness, & Future Threats to Humanity - Jacob Morgan (thefutureorganization.com)