Selected Papers

I list below (with a short description) my best published/accepted works - quite a few of them being with my own students at the University of Michigan. Each of these following works - built upon rigorous decision science, data science, and/or economics methodologies, are characterized by (a) a strong engineering and/or business significance, (b) a mathematically rigorous problem formulation, (c) formally provable solutions often validated by real-world data, (d) a socially impactful solution message, and (e) inter-disciplinary and multi-university research collaborations. I am the research lead in most of the publications. For a complete list of my (~50) publications, please request my complete CV.

R. Pal, K. Psounis, J. Crowcroft, F. P. Kelly, P. Hui, J. Kelly, A. Chatterjee, A. Kumar, L. Golubchik, S. Tarkoma, N. Sastry, and B. Nag: When Are Cyber Blackouts in IT-Driven Service Networks Likely?: A Network Oblivious Theory of Cyber R(e)Insurance Feasibility. In ACM Transactions on Management Information Systems, 2020, Vol. 11(4) [Invited Talk at INFORMS Security Conference, 2020, INFORMS Business Analytics Conference, 2020] [Media Mention in Forbes, KCL Spotlight, U.Helsinki News]

Short Description - Nobody likes to think of ‘Black Swan’ events – catastrophes where the end-to-end delivery of some functionality completely fails. However, in today’s increasingly interconnected world, with complex supply-demand obligations among various interacting players, this is an important risk that we must prepare for. For instance, even something as simple as video streaming can involve complex interactions among several entities, including a content provider such as YouTube, BBC or Netflix, a cloud provider or content delivery network that hosts and delivers their content, and several interconnected ISPs that connect the content servers to the end users. Whereas a failure to watch the next cute cat video is unlikely to be a major disaster, as we move towards supporting more complex mission critical applications such as IoT-driven smart cities, connected cars or remote surgery, it is important to understand how failure of different interconnected and networked entities will affect the service delivery of such applications. What we show is that under a wide range of model parameters, the eventual ecosystem-wide effect of cyber blackouts can be limited and contained. Thus the downside of risk to cyber insurance providers may also be manageable. Our results will provide confidence to risk-averse cyber-risk managers such as insurance and re-insurance companies to expand their businesses in the modern cyber-era. This will positively contribute to a society’s economic security, psychological well-being, and global cyber-security in general, and even in extreme scenarios of catastrophic cyber-wartimes. On a more technical note, our results allow cyber-risk managers to safely infer the economic impact of cyber-attacks even in situations when they are not completely aware of the supply-chain nature of service industries, or the mathematical properties of cyber-attacks hitting a supply chain network. To the best of our knowledge, such an impactful mathematically provable result using minimal service network and cyber-attack information is absent in the literature of cyber-insurance, be it for catastrophic cyber-attacks or otherwise – our efforts are completely new and eye-opening.

R. Pal*, Z. Huang*, S. Lototsky, X. Yin, J. Crowcroft, M. Liu, S. De, N. Sastry, and S. Tarkoma: Aggregate Cyber-Risk Management in the IoT Age: Cautionary Statistics for (Re)Insurers and Likes. In IEEE Internet of Things Journal, 2021, Vol. 8(9) [Early version in the Winter Simulation Conference, 2020; Contributed Talk at INFORMS Annual Meeting, 2020]
R. Pal*, Z. Huang*, S. Lototsky, X. Yin, J. Crowcroft, M. Liu, S. De, N. Sastry, and B. Nag: Will Catastrophic Cyber-Risk Aggregation Thrive in the IoT Age?: A Cautionary Economics Tale for (Re)Insurers and Likes. In ACM Transactions on Management Information Systems, 2021, Vol. 12(2)

Short Description [bundling 2 & 3] - According to Pascal Millaire, a cyber-insurance expert and the CEO of CyberCube and ex-VP of Symantec Cyber-Insurance, the insurance industry has a long way to go to meet cyber coverage business interruption needs. He quotes ''I think today we’re only scratching the surface when it comes to the need for business interruption cover in the cyber space”, agreeing that the IoT will push the issue to a crisis point. The IoT age will unlock over USD 10 trillion in new economic activity globally with a projected 50 billion IoT devices to be in operation by 2025. While many of these devices will work well and benefit society, many will be vulnerable even to naive cyber-attacks, leave alone sophisticated modern cyber-crimes such as DDoS, APT, and ransomware attacks. To make things worse, due to common software platforms and applications running on many of these devices, the attack-impact effects will be correlated in time and space. Consequently, when these fail, it will not be unreasonable to expect significant aggregate cyber-loss impacts with statistically very heavy tails - simply because with increasing inter-connectivity, the cost of a failed connection goes higher. According to data released by UK regulatory authorities in 2019, enterprise-related cyber-losses are ranked higher in the country when compared to flood, Japan earthquakes, and most other catastrophes other than California earthquake and Atlantic hurricane-related losses. However, there is a certain likelihood that the nature of a successful attack is of such magnitude that an organizational client's insurance provider is not able to cover the multi-party aggregate losses incurred upon itself by its clients and their descendants in the supply chain thereby needing to re-insure itself via other cyber-insurance firms. To this end, one question worth investigating in the first place is whether an ecosystem comprising a set of profit-minded cyber-risk managers (e.g., cyber-insurance companies), each capable of providing risk aggregation services (e.g., cyber re-insurance) for a service-networked IT environment, is feasible to cover the aggregate cyber-losses arising due to a cyber-attack. Our study focuses on an empirically interesting case of (extreme) heavy tailed cyber-risk distributions that might presenting themselves to cyber-insurance firms in the modern Internet age in the form of catastrophic service disruptions, and could be a possible standard risk distribution to deal with in the near IoT age. We provide a rigorous general theory, rooted in economics [2] and statistics [3], to elicit conditions on (tail-dependent) heavy-tailed cyber-risk distributions under which a risk management firm might find it (non)sustainable to provide aggregate cyber-risk coverage services like re-insurance for IoT-driven societies. Success in realizing profitable cyber re-insurance markets will improve cyber-security alongside boosting and scaling cyber-risk managment businesses, whereas a failure to achieve working re-insurance markets in critically demanding situations post catastrophic cyber-risk events will strongly call for centralized government regulatory action/intervention to promote risk sharing through re-insurance activities for the benefit of service-networked societies in the IoT age, including mitigating intangible psychological costs borne by society individuals and organizational stakeholders in the form of depressed and affected lifestyles.

R. Pal, L. Golubchik, K. Psounis, and P. Hui: Differentiated Security Pricing as an Enabler of Cyber-Insurance - A First Look at a Markets Approach. In IEEE Transactions of Dependable and Secure Computing, 2019, Vol. 16(2)

Short Description - Despite the promising potential of network risk management services (e.g., cyber-insurance) to improve information security, their deployment is relatively scarce, primarily due to such service companies being unable to guarantee profitability. As a novel approach to making cyber-insurance services more viable (and consequently make cyber-space more secure), we explore a symbiotic relationship between security vendors (e.g., Symantec) capable of price differentiating their clients, and cyber-insurance agencies having possession of information related to the security investments of their clients. The goal of this relationship is to (i) allow security vendors to price differentiate their clients based on security investment information from insurance agencies, (ii) allow the vendors to make more profit than in homogeneous pricing settings, and (iii) subsequently transfer some of the extra profit to cyber-insurance agencies to make insurance services more viable. In this paper, we theoretically investigate whether it is feasible to have an efficient market for differentiated security product pricing, primarily with a view to ensuring that security vendors (SVs) make more profit in the differentiated pricing case as compared to the case of non-differentiated pricing. More specifically, we are interested to design practical, novel, and computationally efficient consumer differentiated pricing mechanisms for SVs based on (i) the market structure, (ii) practical communication network structure of SV consumers captured via a consumer’s Bonacich centrality in the network, and (iii) security investment amounts made by SV consumers. To this end, we provide a rigorous theoretical framework for monopoly SV settings, and simulate an oligopoly market environment. To the best of knowledge, the proposed differentiated pricing framework is the first of its kind in the security products domain.

R. Pal and V. Prasanna: The STREAM Mechanism for CPS Security - The Case of the Smart Grid. In IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2017, Vol. 36(4)

Short Description - Cyber-physical systems (CPSs) integrate computation, communication, and physical capabilities to interact with the physical world and humans. In this paper, we develop a novel strategic resource availability management (STREAM) system to improve information integrity and availability in an energy constrained CPS environment under the presence of malicious adversaries. The term “resource” here can be any component of a CPS. The main elements of STREAM are: 1) difficult but realistic “repeated (adversary-defender) game” settings and 2) a set of provably optimal defender strategies plus effective heuristics, against equally potent adversary moves. STREAM is based on the concept of dynamic games in sequential game theory, and is the first system to incorporate the realistic behavioral aspect that in many CPSs, both, the class of adversaries, as well as the class of CPS protectors, could move in a covert and stealthy manner in order to outwit the other in the war on “resource control.” The main question we ask in this work is: how much more effective can a dynamic strategic approach that mimicks real industry strategy under resource/budget constraints to securing CPS systems when compared to a non-strategic approach? In order to demonstrate the effectiveness of STREAM strategies to improve CPS resource availability to the nonadversary, we first conduct a thorough theoretical analysis on a model smart grid CPS as a representative example of a CPS. We then follow up the analysis with an extensive simulation study on the standard IEEE 14 smart power grid architecture. The results show that STREAM strategies improve smart grid system integrity and availability by approximately upto 67% when compared to non-strategic approaches. Our proposed (analysis, simulation) suite for the grid is extendible to general CPS application domains.

R. Pal, L. Golubchik, K. Psounis, and T. Bandyopadhyay: On Robust Estimates of Correlated Cyber-Insured IT Risk: A First Take of Optimal AI-Based Estimates under 'Small' Data. In ACM Transactions on Management Information Systems, 2019, Vol. 10(3).

Short Description - A major outcome of statistics and AI-based methodologies towards assessing cyber-risk in IT firms (e.g., via an audit process) is the evaluation of a joint distribution of multiple risk (random) variables (e.g., quality of antivirus, frequency of monitoring, etc.) coming from heterogeneous distribution families. As a major modeling drawback, prior work in closed-form IT cyber-risk quantification research assumed that any pair of cyber-risk random variables in a probabilistic graphical model (e.g,, a Bayesian network), used as a formal buidling block to evaluate joint-distributions, are linearly correlated with each other. This simplistic assumption might not always hold true for general IT organizational environments. Thus, the use of existing tools in general will result in loose estimates of correlated IT risk and will subsequently affect cyber-insurance companies in framing profitable coverage policies for IT organizations. The use of statistical copulas mitigates the issue of accurately accounting for non-linear correlations between cyber-risk variables during joint distribution computation - however are too computationally expensive to evaluate at scale. On the other hand Bayesian networks ease the burden of joint distribution computation via their conditional independence constructs - however, are too restricted to derive closed form expressions for cyber-risk variables of arbitrary distributions. The question worth investigating is: is it possible to get the best of both worlds? - as in, merge the power of copulas with that of the applicability of conditional independence, to devise a novel copula-driven probabilistic graphical model that computationally efficiently evaluates a joint cyber-risk distribution. To this end, we provide a rigorous theoretical framework that answers this question in the affirmative, and as a bonus feature, does not require a lot of empirical samples to converge to the true correlation value between random variables.

R. Pal, L. Golubchik, K. Psounis, and P. Hui: Will Cyber-Insurance Improve Network Security? A Market Analysis. In Proceedings of IEEE INFOCOM, 2014, Toronto, Canada. [An early Coase bargaining friendly version appeared as a regular paper in IEEE ICDCS, 2010; An extension appeared in a regular paper of ACM SIGMETRICS PER, Vol. 45(4), 2018; An invited entry for the (Springer) Encyclopedia of Crytography, Security, and Privacy, July 2022], [Media Mention of PhD Work in Forbes#1, Forbes#2, the USC News]

Short Description - Recent work in security has illustrated that solutions aimed at detection and elimination of security threats alone are unlikely to result in a robust cyberspace. As an orthogonal approach to mitigating security problems, some have pursued the use of cyber-insurance as a suitable risk management technique. Such an approach has the potential to jointly align with the incentives of security vendors (e.g., Symantec, Microsoft, etc.), cyber-insurers (e.g., ISPs, cloud providers, security vendors, etc.), regulatory agencies (e.g., government), and network users (individuals and organizations), in turn paving the way for comprehensive and robust cyber-security mechanisms. To this end, in this work, we are motivated by the following important question: can cyber-insurance really improve the security in a network? To address this question, we adopt a market-based approach. Specifically, we analyze regulated monopolistic and competitive cyber-insurance markets, where the market elements consist of risk-averse cyber-insurers, risk-averse network users, a regulatory agency, and security vendors. Our results show that (i) without contract discrimination amongst users, there always exists a unique market equilibrium for both market types, but the equilibrium is inefficient and does not improve network security, and (ii) in monopoly markets, contract discrimination amongst users results in a unique market equilibrium that is efficient, which in turn results in network security improvement - however, the cyber-insurer can make zero expected profits. The latter fact is often sufficient to de-incentivize the insurer to be a part of a market, and will eventually lead to its collapse. This fact also emphasizes the need for designing mechanisms that incentivize the insurer to permanently be part of the market.

R. Pal*, J. Li*, J. Crowcroft,, M. Liu, Y. Li, and N. Sastry: Privacy Risk is a Function of Information Type: Learnings for the Surveillance Capitalism Age. In IEEE Transactions on Network and Service Management, 2021, Vol. 18(3) [Extension of our competitive personal data trading ideas presented in ACM SIGCOMM CCR, Vol. 49(3), 2019]

Short Description - Mobile in-app advertising is a multi-billion dollar industry that is an essential part of the current digital ecosystem, and is amenable to sensitive consumer information often being sold downstream without the knowledge of consumers, and in many cases to their annoyance. While this practice, in cases, may result in long-term benefits for the consumers, it can result in serious information privacy (IP) breaches of very significant impact (e.g., breach of genetic data) in the short term - primarily due to statistical correlations between consumer data. The question we raise in our research is: does the type of information being traded downstream play a role in the degree of IP risks generated?. We investigate two general (one-many) information trading market structures between a single data aggregating seller (e.g., enterprise app) and multiple competing buyers (e.g., ad-networks, retailers), distinguished by mutually exclusive and privacy sanitized aggregated consumer data (information) types: (i) data entailing strategically complementary actions among buyers and (ii) data entailing strategically substituting actions among buyers. Our primary question of interest here is: trading which type of data might pose less information privacy risks for society? To this end, we conduct a rigorous microeconomic analysis to arrive at answers to this question. Through our study, we showcase the optimal management of tradeoffs between information privacy (IP) welfare achieved in society and the commercial gains leveraged by the data selling services, as a function of the economic type of data being traded.

C-L. Chen, R. Pal, and L. Golubchik: Achieving Transparency Report Privacy in Linear Time. In ACM Journal of Data and Information Quality, 2022, Vol. 14(2).

Short Description - Providing an algorithmic transparency report (ATR) by data controllers and third party regulatory agencies to decision-facing individuals is one way to investigate whether decisions made in a blackbox are fair and transparent - an immediate application area of considerable social impact being explainable AI for medical diagnoses to enable medical personnel better understand and interpret diagnostic reports, and justify vulnerabilities of deployed AI models through domain expertise. This is a popular topic in research and there have been works in the last decade that have developed methodologies to reduce opaqueness in decision making and improve on its fairness relative to certain protected attributes. The notion of transparency has also made its way into recently implemented policies for data protection such as in the EU General Data Protection Regulation (GDPR), and the California Consumer Privacy Act of 2018 (CCPA or AB-375) - both of which regulates the processing of collected personal or non-personal data of any data subject (the natural person to whom the data and the decision process relate). An accountable algorithmic transparency report (ATR) should ideally investigate the (a) transparency of the underlying algorithm, and (b) fairness of the algorithmic decisions, and at the same time preserve data subjects' privacy. However, a provably formal study of the impact to data subjects' privacy caused by the utility of releasing an ATR (that investigates transparency and fairness), is yet to be addressed in the literature - a research challenge we take up in this work. The far-fetched benefit of such a study lies in the methodical characterization of privacy-utility trade-offs for release of ATRs in public, and their consequential application-specific impact on the dimensions of society, politics, and economics. To this end, we first explicitly demonstrate inference attacks on data subjects' private information using a real dataset and show that such attacks can be performed on various transparency schemes without strong assumptions of adversaries' knowledge. To preserve data subjects' privacy, we then propose a linear-time optimal-privacy scheme, built upon standard linear fractional programming (LFP) theory, for announcing ATRs, subject to constraints controlling the tolerance of privacy perturbation on the utility of transparency schemes.

R. Pal*, Y. Wang*, C. Light*, Y. Dong*, P. Ghosh*, M. Liu, H. Nagubandi*, L. Golubchik, S. De*, and B. Nag*: Do People Favor Personal Data Markets in a Surveillance Society?: The Behavioral Economics in an Unequal Economy. In Winter Simulation Conference, Phoenix, Arizona, USA, 2021 [A related concept presented in ACM SIGCOMM CCR, Vol. 49(3), 2019;][Media Mention in The Economic Times][Media Mention in Forbes (#1), Forbes (#2)][Media Mention in The Financial Express#1 The Financial Express#2]

Short Description - In the year 2016, Meta (Facebook) publicly released a set of 98 personal data points that they collect about their users. These “98 dimensions” span the interests of most users of social platforms, anywhere on the globe, and potentially reveal private and psychological traits. A primary commercial reason for OSAs to collect such data is targeted advertising that also improves the user experience as it suggests products and services of interest. Market data suggests that more than 95% of Facebook’s revenue comes from advertising, and this business model can only be successful if the adverts are successful too. Recent studies in the western world show a diverse range of the monetary impact of personal data (PD) on the earnings of e-commerce firms and society individuals. As an example, the absence of a single attribute like ZIP code in a customer's data record (let alone the entire record) collected by a frequently used popular service (e.g., Amazon shopping) can result in an opportunity cost of up to US$250 for the service providing firm. More importantly, frequently collected personal data by e- commerce firms such as age, sex, browsing activity, geo-location, etc., of the average (mobile-savvy) individual is worth at least US$1000 annually. Such an amount if paid to OSA users (after PPP conversion) as a cash payment for their PD can significantly reduce the GDP-induced macro-economic inequality in any particular economy around the globe - its effect most likely to be felt on economies with high inequality - e.g., for the smart-phone penetrating, GDP-rich, but highly inequal Indian economy, a PPP-converted value of US$1000 is likely to reduce the average economic inequality by one-third. In particular, the question we seek to address in this paper is: are individuals willing to receive direct payment in the first place? On the one hand, it might seem obvious that users would welcome the opportunity for financial gain. On the other hand, such direct payment likely will necessitate the explicit specification of what is being transacted (quantity and quality), on what terms, to whom and for what purpose -- what if users will have second thoughts about their OSA use when presented with such information, which was previously opaque and vague? But, would it be just as likely that such information gives the users a sense of agency and control, and that if they are given the opportunity to make the explicit and affirmative decision of selling their data for direct monetary gain, then their concerns over privacy risk (along with adverse future repercussions of algorithmic bias from AI-engines) could be substantially alleviated? Of course, it could be that both are true but each for a different segment of the population. More precisely, the basic question we aim to investigate is: will a transparent personal data economy (TPDE) that commensurately reimburses OSA users for their PD tilt the mindset of these users in favor of sharing PD with online social applications? One of our initial experimental efforts directly related to our research motivation consists of the first large-scale non-online pilot RCTs (as part of an inter-continental collaboration) on approximately 22500 individuals in India prior to COVID, to assess the personal data trading preference/ opinion of citizens in a transparent PD economy. A primary starting goal was to characterize the statistical nature of the preference distribution for a large, diverse, and high-GDP (nominal) country like India. We discover (and proved through statistical theory) that the distribution is heavy-tailed and might be weakly scale-free in nature, with more than 50% of the surveyed subjects quite keen to embrace TPDEs. In addition, we provide behavioral economic justifications for observed subject behavior.

R.Pal, P. Hui, and V. Prasanna - On Privacy Engineering in the Smart Micro-Grid. In IEEE Transactions on Knowledge and Data Engineering, 2018, Vol. 31(5)

Short Description - In developing countries, reliable electricity access is often undermined by the absence of supply from the national power grid and/or load shedding. To alleviate this problem, smart micro-grid (SMG) networks that are small scale distributed electricity provision networks composed of individual electricity providers and consumers, are being increasingly deployed. To ensure the reliable operation of SMGs, monitoring is necessary for data collection and state estimation processes. However, highly calibrated and trustworthy smart meters that are ideally suited to perform such monitoring tasks are often costly and non-ideally suited to SMGs which operate under unreliable communication network infrastructures. As a result, SMGs are an easy target to an adversary who can very easily gain access to private information by monitoring transmission between nodes in the SMG network, and launch inference-based privacy attacks. These attacks lead to electricity theft and grid instability problems in the SMG. The widely popular differential privacy (DP) technique (a rigorous technique in the family of privacy-preserving data publishing (PPDP) techniques to mathematically guarantee the preservation of data privacy) does not address multi-attribute correlations, that are inherently exploited by an adversary in inference attacks. In this paper, we propose HIDE, an oblivious computationally efficient, and rigorous information-theoretic privacy engineering framework for datasets/databases arising in the SMG environments that robustly accounts for multi-attribute correlations while preserving data privacy in a provably optimal fashion. A salient and powerful advantage of HIDE is its ability to generate optimal utility-privacy tradeoffs (computationally efficiently) when the privacy preserving entity in the worst case might have no prior statistical information that links a user's private data with his public data.

R. Pal*, Y. Wang*, J. Crowcroft,, M. Liu, S. De, S. Tarkoma, Y. Li, A. Kumar, P. Hui, and B. Nag: Preference-Based Privacy Markets. In IEEE Access, 2020, Vol. 8 [Extension of our competitive personal data trading ideas presented in ACM SIGCOMM CCR, Vol. 49(3), 2019; Contributed Talk at INFORMS Security, 2020][Media Mention in The Economic Times]

Short Description - In the modern era of the mobile apps (the era of surveillance capitalism - as termed by Shoshana Zuboff) huge quantities of surveillance data about consumers and their activities offer a wave of opportunities for economic and societal value creation. ln-app advertising - a multi-billion dollar industry, is an essential part of the current digital ecosystem driven by free mobile applications, where the ecosystem entities usually comprise consumer apps, their clients (consumers), ad-networks, and advertisers. Sensitive consumer information is often being sold downstream in this ecosystem without the knowledge of consumers, and in many cases to their annoyance. While this practice, in cases, may result in long-term benefits for the consumers, it can result in serious information privacy breaches of very significant impact (e.g., breach of genetic data) in the short term. The question we raise through this paper is: is it economically feasible to trade consumer personal information with their formal consent (permission) and in return provide them incentives (monetary or otherwise)?. In view of (a) the behavioral assumption that humans are `compromising' beings and have privacy preferences, (b) privacy as a good not having strict boundaries, and (c) the practical inevitability of inappropriate data leakage by data holders downstream in the data-release supply-chain, we propose a design of regulated efficient/bounded inefficient economic mechanisms for oligopoly data trading markets using a novel preference function bidding approach on a simplified sellers-broker market. Our methodology preserves the heterogeneous privacy preservation constraints (at a grouped consumer, i.e., app, level) upto certain compromise levels, and at the same time satisfies information demand (via the broker) of agencies (e.g., advertising organizations) that collect client data for the purpose of targeted behavioral advertising.

R. Pal* and P. Hui: Economic Models for Cloud Service Markets: Pricing and Capacity Planning. In Theoretical Computer Science (TCS), 2013, Vol. 96 [Received an honor citation from TCS for being one of their top FIVE accessed papers from 2010-2014. This work was done in the industry (T-Labs, Berlin) is probably the first attempt to formally address cloud service economics, and a likely the first systems application of compute-efficient queueing games.]

Short Description - Cloud computing is a paradigm that has transformed and revolutionized the next generation IT industry by making software available to end-users as a service. Multiple public cloud providers (e.g., Amazon, Google) co-existing in a cloud computing market provide similar services (software as a service) to its clients, both in terms of the nature of an application, as well as in quality of service (QoS) provision. The decision of whether a cloud hosts (or finds it profitable to host) a service in the long-term would depend jointly on the price it sets, the QoS guarantees it provides to its customers, and the satisfaction of the advertised guarantees. In this research effort, we first investigate the following fundamental question: how can multiple competing cloud providers in a market set their service prices and QoS (SLA) levels to sustain a profitable business? To answer this question, we devise and analyze three inter-organizational economic models relevant to cloud networks. We formulate our problems as non cooperative and queueing-driven price and QoS games between multiple cloud providers existing in a cloud market to investigate the existence, uniqueness, and computation of market equilibria. A cloud provider services end-user requests on behalf of cloud customers, and due to the uncertainty in user demands over time, tend to over-provision resources like CPU, power, memory, storage, etc., in order to satisfy QoS guarantees. As a result of over-provisioning over long timescales, server utilization is very low and the cloud providers have to bear unnecessarily wasteful costs. In this regard, the price and QoS levels set by the CPs drive the end-user demand, which plays a major role in CPs estimating the minimal capacity to meet their advertised guarantees. By the term ‘capacity’, we imply the ability of a cloud to process user requests, i.e., number of user requests processed per unit of time, which in turn determine the amount of resources to be provisioned to achieve a required capacity. As a second important goal in our research, we address the capacity planning/optimal resource provisioning problem in single-tiered and multi-tiered cloud networks using a queueing networks approach. We develop, analyze, and compare models that cloud providers can adopt to provision resources in a manner such that there is minimum amount of resources wasted, alongside QoS guarantees being satisfied.

R. Pal*, N. Sastry, E. Obiodu**, S. Prabhu**, and K. Psounis: EdgeMart - A Networked OTT Economy on the Wireless Edge for Saving Multimedia IP Bandwidth. Under Revision, 2021 [Contributed Talk at INFORMS Annual Meeting, 2020]

Short Description - With the advent of 5G+ services, it has become increasingly convenient for mobile users to enjoy high quality multimedia content from CDN driven streaming and catch-up TV services (Netflix, iPlayer) in the (post-)COVID over-the-top (OTT) content rush. To relieve ISP owned fixed-line networks from CDN streamed multimedia traffic, the Wi-Stitch system was recently proposed to (a) leverage 5G services and enable consumers to share cached multimedia content at the edge, and (b) consequently, and more importantly, reduce IP traffic at the core network. Unfortunately, given that contemporary multimedia content might be a monetised asset, the ideal Wi-Stitch architecture does not take this important fact into account for shared content. The question that motivates us is: can we have an implementation efficient licensed content sharing economy at the wireless edge to un-burden the Internet core of multimedia IP traffic? We present EdgeMart - a futuristic, content provider federated, and computationally sustainable networked (graphical) market economy for paid-sharing of cached licensed (OTT) content with edge users of a Wi-Stitch wireless edge network (WEN). EdgeMart is a unique oligopoly multimedia market (economy) that comprises competing networked sub-markets of non-cooperative content sellers/buyers - each sub-market consisting of a single buyer connected (networked) to only a subset of sellers. We prove that for any WEN-supported supply-demand topology, an EdgeMart equilibrium exists that is (a) nearly efficient, (b) robust to edge user entry/exit, and (c) can be reached in poly-time. In addition, we experimentally show that for physical WENs of varying densities, a rationally selfish EdgeMart economy induces similar orders of multimedia IP traffic savings when compared to the ideal (relatively less practical), altruistic, and non-monetized ``economy" implemented on the Wi-Stitch architecture. Moreover, the EdgeMart concept helps envision a regulated edge economy of opportunistic (pay per licensed file) client services for commercial OTT platforms.

R. Pal*, S-H. Lin, A. Ahuja, N. Jagadeesan, A. Kumar, and L. Golubchik: Are Federated Cloud Sharing Systems Sustainable?: On Dynamic Sharing Markets and their Stability. In IEEE Transactions on Sustainable Computing, 2020, Vol. 5(4) [A regular paper in ACM SIGMETRICS PER, Vol. 46(4), 2020; Early version in IEEE ICDCS 2017]

Short Description - Small ‘boutique’ clouds challenging the big three (AWS, Azure, Google Cloud) on speed, cost, flexibility, on-prem, and hybrid cloud options are slowly on the rise. Their recent emergence, both in concept and in practice, has been driven mainly by issues related to service cost and complexity of commercial cloud providers employing massive data centers. However, the resource inelasticity problem faced by the SCs due to their relatively scarce resources might lead to a potential degradation of customer QoS and loss of revenue. A proposed solution to this problem recommends the federated sharing of resources between competing SCs to alleviate the resource inelasticity issues that might arise. Based on this idea, a recent effort proposed SC-Share, a performance-driven static market model for competitive small cloud environments that results in an efficient market equilibrium jointly optimizing customer QoS satisfaction and SC revenue generation. However, an important question with a non-obvious answer still remains to be answered, without which SC sharing markets may not be guaranteed to sustain in the long-run - is it still possible to achieve a stable market efficient state when the supply of SC resources is dynamic in nature? In this research effort, we take a first step to addressing the problem of efficient market design for single SC resource sharing in dynamic environments. We answer our previous question in the affirmative through the use of Arrow and Hurwicz's disequilibrium process in economics, and the gradient play technique in game theory that allows us to iteratively converge upon efficient and stable market equilibria. While this result guarantees a unique, stable, and efficient sharing equilibrium, it falls short of providing a microscopic view into practically likely sharing network structures (graphs) among SC providers in the market, and their effect on market efficiency. After all, not every small cloud would want to share resources (even if available) with other small clouds, as a social constraint. Consequently, in a follow up short extension of IEEE TSUSC, Vol.5(4) to the same journal, we envision a graphical (networked) federated cloud sharing market and ask the following questions: (i) does a market equilibrium exist? and (ii) is market efficiency guaranteed at a market equilibrium? We prove that while a symmetric graphical small cloud resource sharing economy is pure-strategy efficient, an asymmetric one generates inefficiencies that improve with an increased number of SCs in the sharing market. The stability of market equilibrium in this networked economy setting follows from the analysis in IEEE TSUSC, Vol.5(4).

R. Pal*, X. Yin, and L. Golubchik: Graphical Federated Cloud Sharing Markets. In IEEE Transactions on Sustainable Computing, 2020, Vol. 6(4) [A Networked Extension of IEEE Transactions on Sustainable Computing, 2020, Vol. 5(4)]

Short Description - While this result in IEEE TSUSC, Vol. 5(4) guarantees a unique, stable, and efficient sharing equilibrium, it falls short of providing a microscopic view into practically likely sharing network structures (graphs) among SC providers in the market, and their effect on market efficiency. After all, not every small cloud would want to share resources (even if available) with other small clouds, as a social constraint. Consequently, in a follow up short extension of IEEE TSUSC, Vol. 5(4) to the same journal, we envision a graphical (networked) federated cloud sharing market and ask the following questions: (i) does a market equilibrium exist? and (ii) is market efficiency guaranteed at a market equilibrium? We prove that while a symmetric graphical small cloud resource sharing economy is pure-strategy efficient, an asymmetric one generates inefficiencies that improve with an increased number of SCs in the sharing market. The stability of market equilibrium in this networked economy setting follows from the analysis in IEEE TSUSC, Vol.5(4).

R. Pal*, C. Chelmis, M. Frincu, and V. Prasanna: MATCH for the Prosumer Smart Grid: The Algorithmics of Real-Time Power Balance. In IEEE Transactions on Parallel and Distributed Systems, 2016, Vol. 27(12)

Short Description - Prosumers or proactive consumers are steadily on the rise in emerging Smart Grid systems. These consumers, apart from their traditonal role of using energy from the grid, also are actively involved in individually transferring stored energy from renewable sources such as wind and solar, to the grid. The large-scale integration of renewable generation in the emerging grid will re-define ways of meeting consumer energy demands, and more importantly drive greener and cost-effective utility operations. In this paper, we investigate the following question: can we effectively match consumer demand with the grid supply in real-time, and in the presence of renewables? We formulate this question as a stochastic optimization problem and propose MATCH, a fast distributed real-time algorithm that accounts for the uncertainties in (i) renewable generation, (ii) the latter's transmission through the grid network, (iii) loads, and (iv) energy prices, and balances power in the Smart Grid at all times. MATCH is based on the Lyapunov stochastic optimization framework and scales to localities with a large number of networked renewable generation sources. Through our study, we showcase that MATCH outputs unique approximate-optimal grid parameter configuration vectors in real-time that ensure perennial supplydemand balance in the grid at a minimum cost.

S-H. Lin*, R. Pal*, B. Wang, and L. Golubchik: On a Market-Driven Hybrid P2P Video Streaming Approach. In IEEE Transactions on Multimedia, 2016, Vol. 19(5). [Early version in ACM/IEEE IWQoS, 2015]

Short Description - Consistent (pause-free) quality of service is required in peer-to-peer (P2P) video streaming systems. In this research, we aim to eliminate the problem of playback pauses in such systems via the use of positive incentives for peers to contribute high upload rates. We model our problem as a market, where the market stakeholders consist of multiple content providers, advertisement providers, and network peers; the positive incentives for peers in the market are reduced advertisement (ad) viewing durations. From a system design perspective, one of our primary goals is to compute the market equilibria that include appropriate ad viewing durations, offering sufficient incentives for network peers to continue contributing. We take an extensive simulation-based approach to demonstrate that we can mitigate the “playback pause” problem for peers significantly as compared to existing approaches, generate sufficient utility for advertisers to be part of the market, and enable content providers to achieve their desired utility by providing sufficient incentives for all peers to stay in the system without violating ad provider agreements.

R. Pal*, C. Chelmis, M. Frincu, and V. Prasanna: Towards Dynamic Demand Response - On Efficient Consumer Grouping Algorithmics. In IEEE Transactions on Sustainable Computing, 2016, Vol 1(1). [Early version in ACM E-Energy, 2015]

Short Description - The widespread monitoring of electricity consumption due to increasingly pervasive deployment of networked sensors in urban environments has resulted in an unprecedentedly large volume of data being collected. Particularly, with the emerging Smart Grid technologies becoming more ubiquitous, real-time and online analytics for discovering the underlying structure of increasing-dimensional (w.r.t. time) consumer time series data are crucial to convert the massive amount of fine-grained energy information gathered from residential smart meters into appropriate demand response (DR) insights. The relevant question of interest here is: how can we effectively perform such dynamic analytics over time and space? In this paper we propose READER and OPTIC, that are real-time and online algorithmic pre-processing frameworks respectively, for effective DR in the Smart Grid. READER (OPTIC) helps discover underlying structure from increasing-dimensional consumer consumption time series data in a provably optimal real-time (online) fashion. READER (OPTIC) catalyzes the efficacy of DR programs by systematically and efficiently managing the energy consumption data deluge, at the same time capturing in real-time (online), specific behavior, i.e., households or time instants with similar consumption patterns. The primary feature of READER (OPTIC) is a real-time (online) randomized approximation algorithm for grouping consumers based on their electricity consumption time series data, and provides two crucial benefits: (i) time efficiently tackles high volume, increasing-dimensional time series data and (ii) provides provable worst case grouping performance guarantees. We validate the grouping and DR efficacy of READER and OPTIC via extensive experiments conducted on both, a USC microgrid dataset as well as a synthetically generated dataset.

Page updated

Report abuse