Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ZoneMinder collector and alarms #9989

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

pepe386
Copy link

@pepe386 pepe386 commented Sep 25, 2020

Summary

Collector to monitor zoneminder cameras. ZoneMinder (https://zoneminder.com/) is an open source video surveillance software system. Two alarms are included as well.

Component Name

New Python collector module.

Test Plan

I am currently running the collector on my server and I can test after merge as well.

Additional Information

@github-actions github-actions bot added area/build Build system (autotools and cmake). area/collectors Everything related to data collection area/docs area/external/python area/health labels Sep 25, 2020
@lgtm-com
Copy link

lgtm-com bot commented Sep 25, 2020

This pull request introduces 1 alert when merging a7e2793 into 17f1a18 - view on LGTM.com

new alerts:

  • 1 for Unused import

@odyslam
Copy link
Contributor

odyslam commented Sep 25, 2020

Thanks for the PR @pepe386, welcome to the netdata fam 💪

@pepe386
Copy link
Author

pepe386 commented Sep 25, 2020

I included directory "~/" in error message and new line at end of file.

@pepe386 pepe386 requested a review from thiagoftsm Sep 25, 2020
@ktsaou
Copy link
Member

ktsaou commented Sep 28, 2020

Codacy Here is an overview of what got changed by this pull request:

Issues
======
- Added 4
           

Complexity increasing per file
==============================
- collectors/python.d.plugin/zoneminder/zoneminder.chart.py  25
         

See the complete overview on Codacy

@ilyam8
Copy link
Member

ilyam8 commented Sep 29, 2020

Hi @pepe386. Thx for contributing! Give me some time to review the PR, a little bit busy atm 😅

@pepe386
Copy link
Author

pepe386 commented Sep 29, 2020

Hi @pepe386. Thx for contributing! Give me some time to review the PR, a little bit busy atm

No problem , take your time.

@ilyam8
Copy link
Member

ilyam8 commented Oct 30, 2020

@pepe386 let's start from applying PEP8 formatting and removing unnecessary ()

Changes, i dont say let's use exact that format (i applied auto-format), could make it other way, but let's keep line length <= 120.

0 netdata (zoneminder-collector *)$ git diff
diff --git a/collectors/python.d.plugin/zoneminder/zoneminder.chart.py b/collectors/python.d.plugin/zoneminder/zoneminder.chart.py
index 6f7cb4d5..9fb1d46d 100644
--- a/collectors/python.d.plugin/zoneminder/zoneminder.chart.py
+++ b/collectors/python.d.plugin/zoneminder/zoneminder.chart.py
@@ -9,6 +9,7 @@ import time, os.path
 try:
     import requests
     import jwt
+
     HAVE_DEPS = True
 except ImportError:
     HAVE_DEPS = False
@@ -45,40 +46,45 @@ CHARTS = {
     },
 }

+
 def zm_generate_refresh_token(zoneminder_url, zm_user, zm_password, connection_timeout):
     try:
-        post_data=dict()
+        post_data = dict()
         post_data["user"] = zm_user
         post_data["pass"] = zm_password
-        r = requests.post(zoneminder_url + '/api/host/login.json', data=post_data, timeout=connection_timeout, verify=False)
+        r = requests.post(zoneminder_url + '/api/host/login.json', data=post_data, timeout=connection_timeout,
+                          verify=False)
         json_data = r.json()
-        if all (k in json_data for k in ("access_token","refresh_token")):
+        if all(k in json_data for k in ("access_token", "refresh_token")):
             try:
-                token_file = open(os.path.expanduser("~/.zm_token.txt"),'w')
+                token_file = open(os.path.expanduser("~/.zm_token.txt"), 'w')
                 token_file.write("{}|{}".format(json_data["access_token"], json_data["refresh_token"]))
                 token_file.close()
             except IOError:
-                return ("<error>", "Error while writing ~/.zm_token.txt file.")
-            return ("ok", "{}|{}".format(json_data["access_token"], json_data["refresh_token"]))
-        return ("<error>", "Invalid api response when trying to generate new access and refresh tokens: " + r.text)
+                return "<error>", "Error while writing ~/.zm_token.txt file."
+            return "ok", "{}|{}".format(json_data["access_token"], json_data["refresh_token"])
+        return "<error>", "Invalid api response when trying to generate new access and refresh tokens: " + r.text
     except requests.exceptions.RequestException as e:
-        return ("<error>", e)
+        return "<error>", e
+

 def zm_generate_access_token(zoneminder_url, refresh_token, connection_timeout):
     try:
-        r = requests.post(zoneminder_url + '/api/host/login.json?token=' + refresh_token, timeout=connection_timeout, verify=False)
+        r = requests.post(zoneminder_url + '/api/host/login.json?token=' + refresh_token, timeout=connection_timeout,
+                          verify=False)
         json_data = r.json()
-        if ("access_token" in json_data):
+        if "access_token" in json_data:
             try:
-                token_file = open(os.path.expanduser("~/.zm_token.txt"),'w')
+                token_file = open(os.path.expanduser("~/.zm_token.txt"), 'w')
                 token_file.write("{}|{}".format(json_data["access_token"], refresh_token))
                 token_file.close()
             except IOError:
-                return ("<error>", "Error while writing ~/.zm_token.txt file.")
-            return ("ok", json_data["access_token"])
-        return ("<error>", "Invalid api response when trying to generate new access token: " + r.text)
+                return "<error>", "Error while writing ~/.zm_token.txt file."
+            return "ok", json_data["access_token"]
+        return "<error>", "Invalid api response when trying to generate new access token: " + r.text
     except requests.exceptions.RequestException as e:
-        return ("<error>", e)
+        return "<error>", e
+

 class Service(SimpleService):
     def __init__(self, configuration=None, name=None):
@@ -103,79 +109,88 @@ class Service(SimpleService):
         bool_login = True
         disk_space = 0

-        #if user is not defined, then do not attempt to login
+        # if user is not defined, then do not attempt to login
         if not self.zm_user:
             bool_login = False

-        #get access token from file or zoneminder api
+        # get access token from file or zoneminder api
         if bool_login:
             try:
                 token_file = open(os.path.expanduser("~/.zm_token.txt"))
-                access_token,refresh_token = token_file.read().split('|')
+                access_token, refresh_token = token_file.read().split('|')
                 token_file.close()
             except IOError:
-                result,output = zm_generate_refresh_token(self.zoneminder_url, self.zm_user, self.zm_password, self.connection_timeout)
-                if ("<error>" in result):
+                result, output = zm_generate_refresh_token(self.zoneminder_url, self.zm_user, self.zm_password,
+                                                           self.connection_timeout)
+                if "<error>" in result:
                     self.debug("error: " + output)
                     return None
                 self.debug("new access and refresh tokens were generated...")
-                access_token,refresh_token = output.split('|')
+                access_token, refresh_token = output.split('|')

-            #get jwt information
+            # get jwt information
             jwt_access_data = jwt.decode(access_token, verify=False)
             jwt_refresh_data = jwt.decode(refresh_token, verify=False)

-            #get new refresh token if it expires in less than 30 minutes
-            if ( ( jwt_refresh_data['exp'] - time.time() ) < 1800 ):
+            # get new refresh token if it expires in less than 30 minutes
+            if (jwt_refresh_data['exp'] - time.time()) < 1800:
                 self.debug("generating new refresh token...")
-                result,output = zm_generate_refresh_token(self.zoneminder_url, self.zm_user, self.zm_password, self.connection_timeout)
-                if ("<error>" in result):
+                result, output = zm_generate_refresh_token(self.zoneminder_url, self.zm_user, self.zm_password,
+                                                           self.connection_timeout)
+                if "<error>" in result:
                     self.debug("error: " + output)
                     return None
-                access_token,refresh_token = output.split('|')
+                access_token, refresh_token = output.split('|')

-            #get new access token if current token expires in less than 5 minutes
-            if ( ( jwt_access_data['exp'] - time.time() ) < 300 ):
-                result,output = zm_generate_access_token(self.zoneminder_url, refresh_token, self.connection_timeout)
-                if ("<error>" in result):
+            # get new access token if current token expires in less than 5 minutes
+            if (jwt_access_data['exp'] - time.time()) < 300:
+                result, output = zm_generate_access_token(self.zoneminder_url, refresh_token, self.connection_timeout)
+                if "<error>" in result:
                     self.debug("error: " + output)
                     return None
                 access_token = output

-        #get data from monitors api call
+        # get data from monitors api call
         try:
-            r = requests.get(self.zoneminder_url + '/api/monitors.json?token=' + access_token, timeout=self.connection_timeout, verify=False)
+            r = requests.get(self.zoneminder_url + '/api/monitors.json?token=' + access_token,
+                             timeout=self.connection_timeout, verify=False)
             json_data = r.json()
         except requests.exceptions.RequestException as e:
             self.debug(e)
             return None

-        if all (k in json_data for k in ("success","data")):
-            if (json_data['success'] == False and 'Token revoked' in json_data['data']['name']):
+        if all(k in json_data for k in ("success", "data")):
+            if json_data['success'] == False and 'Token revoked' in json_data['data']['name']:
                 self.debug("token was revoked, generating new tokens, will try to collect data in next run...")
-                result,output = zm_generate_refresh_token(self.zoneminder_url, self.zm_user, self.zm_password, self.connection_timeout)
-                if ("<error>" in result):
+                result, output = zm_generate_refresh_token(self.zoneminder_url, self.zm_user, self.zm_password,
+                                                           self.connection_timeout)
+                if "<error>" in result:
                     self.debug("error: " + output)
                 return None

-        if ("monitors" in json_data):
+        if "monitors" in json_data:
             for monitor in json_data["monitors"]:
-                if ("Monitor" in monitor):
+                if "Monitor" in monitor:
                     try:
                         disk_space += float(monitor["Monitor"]["TotalEventDiskSpace"])
                     except Exception as e:
                         self.debug(e)
-                    if (monitor["Monitor"]["Function"] == "None" or monitor["Monitor"]["Enabled"] == "0"):
+                    if monitor["Monitor"]["Function"] == "None" or monitor["Monitor"]["Enabled"] == "0":
                         continue
                     if "zm_fps_" + monitor["Monitor"]["Id"] not in self.charts['camera_fps']:
-                        self.charts['camera_fps'].add_dimension(["zm_fps_" + monitor["Monitor"]["Id"], monitor["Monitor"]["Name"], 'absolute'])
+                        self.charts['camera_fps'].add_dimension(
+                            ["zm_fps_" + monitor["Monitor"]["Id"], monitor["Monitor"]["Name"], 'absolute'])
                     if "zm_bandwidth_" + monitor["Monitor"]["Id"] not in self.charts['camera_bandwidth']:
-                        self.charts['camera_bandwidth'].add_dimension(["zm_bandwidth_" + monitor["Monitor"]["Id"], monitor["Monitor"]["Name"], 'absolute', None, 1024])
+                        self.charts['camera_bandwidth'].add_dimension(
+                            ["zm_bandwidth_" + monitor["Monitor"]["Id"], monitor["Monitor"]["Name"], 'absolute', None,
+                             1024])
                     if "zm_events_" + monitor["Monitor"]["Id"] not in self.charts['events']:
-                        self.charts['events'].add_dimension(["zm_events_" + monitor["Monitor"]["Id"], monitor["Monitor"]["Name"], 'absolute'])
+                        self.charts['events'].add_dimension(
+                            ["zm_events_" + monitor["Monitor"]["Id"], monitor["Monitor"]["Name"], 'absolute'])
                     try:
                         data["zm_fps_" + monitor["Monitor"]["Id"]] = float(monitor["Monitor_Status"]["CaptureFPS"])
-                        data["zm_bandwidth_" + monitor["Monitor"]["Id"]] = float(monitor["Monitor_Status"]["CaptureBandwidth"])
+                        data["zm_bandwidth_" + monitor["Monitor"]["Id"]] = float(
+                            monitor["Monitor_Status"]["CaptureBandwidth"])
                         data["zm_events_" + monitor["Monitor"]["Id"]] = float(monitor["Monitor"]["TotalEvents"])
                     except Exception as e:
                         self.debug(e)

@pepe386 pepe386 requested a review from Ferroin as a code owner Nov 1, 2020
@pepe386 pepe386 requested a review from knatsakis as a code owner Nov 1, 2020
@pepe386 pepe386 requested review from ilyam8 and removed request for prologic Nov 1, 2020
Copy link
Contributor

@DShreve2 DShreve2 left a comment

Small changes requested for readme.

```

## Charts
This module generates 4 charts:
Copy link
Contributor

@DShreve2 DShreve2 Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This module generates 4 charts:
This module generates the following four charts:

zm_pass: '' # the zoneminder password to use
```

"zm_user" and "zm_pass" password can be left blank if authentication is not set. If charts are not shown on netdata dashboard, try restarting netdata:
Copy link
Contributor

@DShreve2 DShreve2 Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"zm_user" and "zm_pass" password can be left blank if authentication is not set. If charts are not shown on netdata dashboard, try restarting netdata:
If authentication is not set, "zm_user" and "zm_pass" password can be left blank. If charts are not shown on Netdata dashboard, try restarting Netdata:

4. Disk usage of all saved events

## Alarms
1. Average FPS of last 5 minutes: by default this alarm sets "warn" level when average fps is less than 5 in the last 5 minutes, and "critical" level when average fps is less than 2 in the last 5 minutes.
Copy link
Contributor

@DShreve2 DShreve2 Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Average FPS of last 5 minutes: by default this alarm sets "warn" level when average fps is less than 5 in the last 5 minutes, and "critical" level when average fps is less than 2 in the last 5 minutes.
1. Average FPS of last 5 minutes: by default this alarm sets "warn" level when average fps is less than five in the last five minutes, and "critical" level when average fps is less than two in the last five minutes.


## Alarms
1. Average FPS of last 5 minutes: by default this alarm sets "warn" level when average fps is less than 5 in the last 5 minutes, and "critical" level when average fps is less than 2 in the last 5 minutes.
2. Time since last successful data collection: "warn" when no data is collected for last 5 minutes, and "critical" when no data is collected for last 15 minutes.
Copy link
Contributor

@DShreve2 DShreve2 Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. Time since last successful data collection: "warn" when no data is collected for last 5 minutes, and "critical" when no data is collected for last 15 minutes.
2. Time since last successful data collection: "warn" when no data is collected for last five minutes, and "critical" when no data is collected for last 15 minutes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build Build system (autotools and cmake). area/collectors Everything related to data collection area/docs area/health collectors/python.d
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants