With the advent of Cloud Era, real-time streaming data processing applications are used more and more widely. As a concrete example, real-time queries on a large number of sensors are highly demanding and needs to be carefully designed. Sensor monitoring analysts are inexperienced on coding techniques in spite of the rapid development of real-time processing platforms and communities. What's more, the state-of-the-art systems lacks support for multi-query processing and sharing of the original data and results with prior information of the queries. To tackle the challenges, we endeavor to design a light-weight but effective scheme to support multiple queries running on a large number of sensors. We focus on aggregations on sliding windows due to its practical importance, and hence design a query language, SAQL, for the professional analysts to construct their own streaming computational logic. We also propose the whole system, SAQS, which can interpret SAQL into stream processing programs and support the efficient execution of multiple queries in a distributed real-time environment. SAQS can detect the data sharing at different granularity and uses a greedy partition algorithm to balance the load among worker nodes. Our extensive experimental evaluations establish the convenience and efficiency of SAQS.
Yumin Zhou, Peng Wang, Wei Wang
2018 5th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)/2018 4th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom)