Research

Recent advances in reasoning capabilities have made large language models (LLMs) promising for safety-critical systems, where a single model serves both hard and soft real-time tasks. However, jointly supporting them is difficult because batched GPU execution can break temporal isolation among tasks, and the autoregressive nature of LLMs makes execution time hard to predict. In this paper, we propose CoRT, a real-time LLM framework that guarantees hard-task deadlines while improving soft-task responsiveness. To satisfy these heterogeneous timing requirements, CoRT regulates LLM execution time by reducing autoregressive computation through three control knobs: early exit, output-length control, and model selection. We implement CoRT in PyTorch and demonstrate that it achieves up to 4.9× higher schedulability for hard tasks and 7× higher responsiveness for soft tasks than prior approaches. To the best of our knowledge, this is the first work that jointly supports hard and soft tasks on batched LLM serving platforms.

The proliferation of mobile and IoT devices has sparked growing interest in multi-device interaction, where a single app operates across multiple devices to leverage their diverse capabilities. However, the current methods of app development and usage remain bound to a single-device paradigm, rendering multi-device apps difficult to implement and deploy. This paper presents COMET, a novel mobile app framework that enables the dynamic distribution of app components across multiple devices at runtime, supporting a wide range of multi-device scenarios, including collaborative applications, smart environments, and interactions across heterogeneous personal devices. COMET enables developers to specify distributable components using lightweight annotations and employs build-time code instrumentation to automate the deployment and execution of selected components onto remote devices. This approach supports multi-device interaction with minimal developer effort and without requiring system-level modifications. To realize this, COMET addresses four key challenges: (i) the static partitioning of app components and extraction of their dependencies at build time, (ii) efficient execution of these components on remote devices, (iii) preservation of intercomponent communication across devices, and (iv) synchronization of distributed components that share a global state. We implemented a prototype of COMET on Android and evaluated it using real-world apps. Our evaluation with eight case-study apps shows that component distribution completes within 251.8 ms. A user study with 15 participants further demonstrates that COMET provides intuitive multi-device interaction with acceptable responsiveness.

Specialized hardware accelerators facilitate timing guarantees for real-time DNNs, but high computational demands cause severe thermal stress. We present SCOUT, a real-time framework for Google Edge TPU that jointly satisfies timing/thermal constraints. The key idea couples task-level dynamic frequency scaling (DFS) with runtime on-chip SRAM allocation to offset DFS-induced performance loss. To this end, SCOUT provides i) a thermal model that reflects SRAM’s thermal impact and ii) an SRAM reallocation technique that safely adjusts allocations while mitigating write overheads. Our experiments show SCOUT schedules up to 46% more DNN task sets than prior approaches, while adhering to thermal constraints.

The advent of large language models (LLMs) has opened up new opportunities in the field of mobile task automation. Their superior language understanding and reasoning capabilities allow users to automate complex and repetitive tasks. However, due to the inherent unreliability and high operational cost of LLMs, their practical applicability is quite limited. To address these issues, this paper introduces MobileGPT, an innovative LLM-based mobile task automator enhanced with a unique app memory. MobileGPT emulates the cognitive process of humans interacting with a mobile app—explore, select, derive, and recall. This approach allows for a more precise and efficient learning of a task’s procedure by breaking it down into smaller, modular components that can be re-used, re-arranged, and adapted for various objectives. We implement MobileGPT using online LLMs services (GPT-3.5 and GPT-4) and evaluate its performance on 50 unique mobile tasks across 5 widely used mobile apps. The results indicate that MobileGPT can adapt learned tasks to varying contexts with 100% accuracy and reduces their latency and cost by 69.22% and 77.36% compared to a GPT-4 powered baseline.

Deep Neural Networks (DNNs) are essential in IoT and CPS applications, with hardware accelerators like Google Edge TPU enhancing performance and energy efficiency. However, unpredictable execution times limit real-time deployment due to B1) deadline-oblivious SRAM allocation and B2) non-preemptive scheduling. Experiments show up to 5.2x execution time variation caused by these issues. To address this, we propose SPET, a system for transparent SRAM allocation and model partitioning. SPET offers i) flexible SRAM allocation without TPU compiler modifications, ii) adaptive model partitioning for predictable preemption, and iii) priority-based online scheduling. Evaluation on TensorFlow Lite with Google Coral shows SPET improves schedulability by up to 79%. A case study on emergency braking in a self-driving car highlights its safety benefits through timely DNN inference.

Mobile apps offer diverse features to enhance user experience, yet users often struggle to use multiple apps simultaneously on small smartphone screens. This paper introduces A-Mash, a mobile platform that simplifies concurrent app interactions to the ease of using a single app. A-Mash allows users to mash up UIs of different apps on one screen based on their preferences by 1) extracting UIs from unmodified apps (dynamic UI extraction), 2) embedding these UIs into a single wrapper app (cross-process UI embedding), and 3) ensuring the process remains transparent to users. To our knowledge, A-Mash is the first system to seamlessly integrate UIs of unmodified legacy apps on a single screen, creating the illusion of a unified app. Potential use cases include creating an all-in-one IoT controller or a personalized news headline collection. A-Mash can also extend to AR, mapping app UIs to physical objects in AR scenes. Our prototype on Android OS shows A-Mash effectively supports UI mashups with minimal performance impact, as validated by user studies in real-world scenarios.

Being able to use a single app across multiple devices can bring novel experiences to the users in various domains including entertainment and productivity. For instance, a user of a video editing app would be able to use a smart pad as a canvas and a smartphone as a remote toolbox so that the toolbox does not occlude the canvas during editing. However, existing approaches do not properly support the single-app multi-device execution due to several limitations, including high development cost, device heterogeneity, and high performance requirement. In this paper, we introduce FLUID-XP, a novel cross-platform multi-device system that enables UIs of a single app to be executed across heterogeneous platforms, while overcoming the limitations of previous approaches. FLUID-XP provides flexible, efficient, and seamless interactions by addressing three main challenges: i) how to transparently enable a single-display app to use multiple displays, ii) how to distribute UIs across heterogeneous devices with minimal network traffic, and iii) how to optimize the UI distribution process when multiple UIs have different distribution requirements. Our experiments with a working prototype of FLUID-XP on Android confirm that FLUID-XP successfully supports a variety of unmodified real-world apps across heterogeneous platforms (Android, iOS, and Linux). We also conduct a lab study with 25 participants to demonstrate the effectiveness of FLUID-XP with real users.

The growing trend of multi-device ownerships creates a need and an opportunity to use applications across multiple devices. However, in general, the current app development and usage still remain within the single-device paradigm, falling far short of user expectations. For example, it is currently not possible for a user to dynamically partition an existing live streaming app with chatting capabilities across different devices, such that she watches her favorite broadcast on her smart TV while real-time chatting on her smartphone. In this paper, we present FLUID, a new Android-based multi-device platform that enables innovative ways of using multiple devices. FLUID aims to i) allow users to migrate or replicate individual user interfaces (UIs) of a single app on multiple devices (high flexibility), ii) require no additional development effort to support unmodified, legacy applications (ease of development), and iii) support a wide range of apps that follow the trend of using custom-made UIs (wide applicability). Previous approaches, such as screen mirroring, app migration, and customized apps utilizing multiple devices, do not satisfy those goals altogether. FLUID, on the other hand, meets the goals by carefully analyzing which UI states are necessary to correctly render UI objects, deploying only those states on different devices, supporting cross-device function calls transparently, and synchronizing the UI states of replicated UI objects across multiple devices. Our evaluation with 20 unmodified, real-world Android apps shows that FLUID can transparently support a wide range of apps and is fast enough for interactive use.

In recent years, the explosion of diverse smart devices such as mobile phones, TVs, watches, and even cars, has completely changed our lives. We communicate with friends through social network services (SNSs) whenever we want, buy stuff without visiting shops, and enjoy multimedia wherever we are, thanks to these devices. However, these smart devices cannot simply interact with each other even though they are right next to each other. For example, when you want to read a PDF stored on a smartphone on a larger TV screen, you need to do complicated work or plug in a bunch of cables. In this paper, we introduce M+, an extension of Android that supports cross-device functionality sharing in a transparent manner. As a platform-level solution, M+ enables unmodified Android applications to utilize not only application functionalities but also system functionalities across devices, as if they were to utilize them inside the same device. In addition to secure connection setup, M+ also allows performing of permission checks for remote applications in the same way as for local. Our experimental results show that M+ enables transparent cross-device sharing for various functionalities and achieves performance close to that of within-device sharing unless a large amount of data is transferred.

In this article, an adaptive real-time communication system is proposed that leverages the software-defined networking (SDN) paradigm to provide end-to-end deadline guarantees. SDN is an emerging networking paradigm that allows control of the network through directly programmable software controllers, offering the flexibility to manage and optimize network resources dynamically. By utilizing these SDN features, a novel networking system, called RT-SDN, is presented that can effectively support real-time communication through cooperative routing and scheduling. To this end, we present a highly efficient routing algorithm that can adaptively reconfigure the routes of existing flows to find bandwidth-guaranteed routes for all flows. We then introduce a new priority assignment scheme that can achieve better deadline guarantees (i.e., better schedulability). In addition, RT-SDN allows routing and scheduling to operate together by employing a feedback loop between them. We implemented a prototype of RT-SDN as an SDN controller and deployed it on a testbed network of 30 BeagleBone devices. Our experiment results show that RT-SDN can be deployed in real-world commodity networks to provide end-to-end deadline guarantees, and the proposed schemes collectively improve the schedulability to a significant degree.