Projects:graph openstack - Revision history

Chentianjia at 08:35, 24 November 2016

2016-11-24T08:35:53Z

Chentianjia: /* 相关资料 */

2016-11-24T08:35:26Z

‎相关资料

Chentianjia at 08:31, 24 November 2016

2016-11-24T08:31:47Z

Chentianjia at 08:21, 24 November 2016

2016-11-24T08:21:27Z

Chentianjia at 08:20, 24 November 2016

2016-11-24T08:20:26Z

Chentianjia at 08:20, 24 November 2016

2016-11-24T08:20:12Z

Chentianjia: Created page with "= 项目名称 = 用Graph的方法检测Openstack故障问题 = 项目介绍 = It is hard to operate and debug systems like OpenStack that integrate many independently develo..."

2016-11-24T08:19:34Z

Created page with "= 项目名称 = 用Graph的方法检测Openstack故障问题 = 项目介绍 = It is hard to operate and debug systems like OpenStack that integrate many independently develo..."

New page

= 项目名称 =
用Graph的方法检测Openstack故障问题

= 项目介绍 =
It is hard to operate and debug systems like OpenStack that integrate many independently developed modules with multiple levels of abstractions. A major challenge is to navigate through the complex dependencies and relationships of the states in different modules or subsystems, to ensure the correctness and consistency of these states. We present a system that captures the runtime states and events from the entire OpenStack-Ceph stack, and automatically organizes these data into a graph that we call system operation state graph. With SOSG we can use intuitive graph traversal techniques to solve problems like reasoning about the state of a virtual machine. Also, using graph-based anomaly detection, we can automatically discover hidden problems in OpenStack. We have a scalable implementation of SOSG, and evaluate the approach on a 125-node production OpenStack cluster, finding a number of interesting problems.

= 参与人员 =

Yong Xiang

Wei Xu

= 相关资料 =

[[File:XY.pdf | 600px ]]

= 项目进展 =
== 2014年11月15日 ==
项目开始

== 2016年8月5日 ==

In Proceedings of ACM SIGOPS Asia-Pacific Workshop on Systems (APSys'16) [BEST PAPER AWARD] Hong Kong, China, 2016

= 相关文献 =

[1] https://www.openstack.org/
[2] https://www.rabbitmq.com/
[3] https://libvirt.org/
[4] http://www.linux-kvm.org/page/Main Page [5] http://openvswitch.org/
[6] http://ceph.com/
[7] Xu W, Huang L, Fox A, et al. “Detecting large-scale system problems by mining console logs[C].” Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles. ACM, 2009: 117-132.
[8] Yuan, Ding, et al. “SherLog: error diagnosis by connecting clues from run-time logs.” ACM SIGARCH computer architec- ture news. Vol. 38. No. 1. ACM, 2010.
[9] Ranking I A A. “Finding Patterns in Static Analysis Alerts[J]”. 2014.
[10] http://neo4j.com/
[11] Gonzalez, Joseph E., et al. “Graphx: Graph processing in a distributed dataflow framework.” 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). 2014.
[12] Turner D, Levchenko K, Snoeren A C, et al. “California fault lines: understanding the causes and impact of network failures[J]”. ACM SIGCOMM Computer Communication Review, 2011, 41(4): 315-326.
[13] Mace, Jonathan, Ryan Roelke, and Rodrigo Fonseca. “Pivot tracing: dynamic causal monitoring for distributed systems.” Proceedings of the 25th Symposium on Operating Systems Principles. ACM, 2015.
[14] Sambasivan R R, Zheng A X, De Rosa M, et al. “Diagnosing Performance Changes by Comparing Request Flows.”.” NSDI. 2011.
[15] Wang C, Viswanathan K, Choudur L, et al. “Statistical tech- niques for online anomaly detection in data centers.” Integrated Network Management (IM), 2011 IFIP/IEEE International Sym- posium on. IEEE, 2011: 385-392.
[16] Solaimani M, Iftekhar M, Khan L, et al. “Statistical technique for online anomaly detection using spark over heterogeneous data from multi-source VMware performance data.” Big Data (Big Data), 2014 IEEE International Conference on. IEEE, 2014: 1086-1094.
[17] Id T, Kashima H. “Eigenspace-based anomaly detection in computer systems[C].” Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2004: 440-449.
[18] Zhao Y, Zheng Z, Wen H. “Bayesian statistical inference in machine learning anomaly detection.” Communications and Intelligence Information Security (ICCIIS), 2010 International Conference on. IEEE, 2010: 113-116.
[19] Dasgupta D, Majumdar N S. “Anomaly detection in multidi- mensional data using negative selection algorithm.” wcci. IEEE, 2002: 1039-1044.
[20] Sun, Yizhou, et al. “Pathsim: Meta path-based top-k similarity search in heterogeneous information networks.” VLDB11 (2011).
[21] Knorr E M, Ng R T, Tucakov V. “Distance-based outliers: algorithms and applications[J].” The VLDB Journal The International Journal on Very Large Data Bases, 2000, 8(3-4): 237-253.
[22] Xie M, Han S, Tian B. “Highly efficient distance-based anomaly detection through univariate with PCA in wireless sensor networks.” Trust, Security and Privacy in Computing and Communications (TrustCom), 2011 IEEE 10th International Conference on. IEEE, 2011: 564-571.
[23] Khoa N L D, Babaie T, Chawla S, et al. “Network anomaly detection using a commute distance based approach[C].” Data Mining Workshops (ICDMW), 2010 IEEE International Conference on. IEEE, 2010: 943-950.
[24] Noble, Caleb C., and Diane J. Cook. “Graph-based anomaly detection.” Proceedings of the ninth ACM SIGKDD interna- tional conference on Knowledge discovery and data mining. ACM, 2003.

@@ Line 14: / Line 14: @@
 = 相关资料 =
-[[File:xy1.jpg | 600px]]
+[[File:xy1.jpg | 800px]]
 [[File:XY.pdf]]

@@ Line 29: / Line 29: @@
 [1] https://www.openstack.org/
-[2] https://www.rabbitmq.com/ //
+[2] https://www.rabbitmq.com/ \\
-[3] https://libvirt.org/ //
+[3] https://libvirt.org/ \\
 [4] http://www.linux-kvm.org/page/Main Page [5] http://openvswitch.org/ //
 [6] http://ceph.com/

@@ Line 4: / Line 4: @@
 = 项目介绍 =
 It is hard to operate and debug systems like OpenStack that integrate many independently developed modules with multiple levels of abstractions.  A major challenge is to navigate through the complex dependencies and relationships of the states in different modules or subsystems, to ensure the correctness and consistency of these states.  We present a system that captures the runtime states and events from the entire OpenStack-Ceph stack, and automatically organizes these data into a graph that we call system operation state graph.  With SOSG we can use intuitive graph traversal techniques to solve problems like reasoning about the state of a virtual machine.  Also, using graph-based anomaly detection, we can automatically discover hidden problems in OpenStack.  We have a scalable implementation of SOSG, and evaluate the approach on a 125-node production OpenStack cluster, finding a number of interesting problems.
 = 参与人员 =
@@ Line 25: / Line 24: @@
 In Proceedings of ACM SIGOPS Asia-Pacific Workshop on Systems (APSys'16) [BEST PAPER AWARD] Hong Kong, China, 2016
 = 相关文献 =