Allow data collecting from LibreQoS into monitoring tools #75

interduo · 2022-09-09T18:07:23Z

I would like to show graphs provided by LibreQoS in our CRM.
@rchac what metod do You prefer to do things like that?

rchac · 2022-09-09T18:16:16Z

I'd recommend to have some script or something to pull stats from InfluxDB, which LibreQoS stores bandwidth data to. It stores data over time so you can neatly graph it easily.

interduo · 2022-09-09T18:20:59Z

Is it possible to retrieve InfluxDB subscriber data by API? If Yes I would make code for generating graphs on CRM side.

rchac · 2022-09-09T20:13:41Z

Yes! https://docs.influxdata.com/influxdb/v2.4/reference/api/

interduo · 2022-09-11T08:18:48Z

@rchac what software do You use to make graphs of retrieved data from API? Or is it possible to retrieve graph image by API?

rchac · 2022-09-11T11:40:48Z

InfluxDB has a UI built-in, so there is often no need to use the API. You can use the influxDBdashboardTemplate.json file included with v1.2 for a pre-configured dashboard.

interduo · 2022-09-11T11:59:26Z

Yes I know but I would like to show graphs inside customer tab in CRM.
I've got this done by now on my current script - it's very fast&useful for debugging customer networking issues.

rchac · 2022-09-11T16:57:19Z

Ah ok gotcha. That is a very good idea for debugging!

interduo · 2022-09-13T17:38:11Z

I dont see option for exporting graphs as an images in influxdb and i think that it would be better to generate graphs in our CRM from retrieved data (because of zooming and other features or things that could be done in future with them).

interduo · 2022-09-22T14:06:35Z

I think that the influxdb (here on QoS Engine) is a bit overfeature (it should be marked as optional)
If I am looking at current resources usage from influxdb - there is almost 7GB - please this is not place in this VM for this.

This would generates more problems, many thoughts like #74 or at least much work to do with some ram usage / disk space monitoring. (more cons than pros in total).

In my opinion we should get opportunity to share circuit traffic data to zabbix/librenms/cacti/sth_else_monitoring_tool and there do the magic (graphing, counting).

For this we need some constant identifier when adding customer - that would not change after reloading, adding, removing customers from beggining of ShapedDevices.csv (not the classid could change when we mix customers in rows, or customer changes it plan).

Then we could prepare gathering data whitch could be used after with:

zabbix_sender (agent installed on LibreQoS for fast sending data to monioring tool),
snmpd
cacti remote poller
or other solutions for network monitoring tool.

This is good by design (graping is getting CPU ussage so the latency would not be touched in any circumstance).
Doing microservices is better than monolith in that case because QoS Engine is almost critical.

Maybe it's a good way to teach: lqTools.py --get-all-circuits-stats with the output of JSON/CSV with:
CircuitID and and all the stats from tc -s -json qdisc
(running 2x8k times stats would be definetly a cpu killer)

interduo · 2022-09-26T14:08:41Z

I see graphBandwidth.py could You add --debug and then just print on the screen the data whitch are transfered to stats?

interduo · 2022-09-26T14:11:05Z

There would be also helpful if we got only error:
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fdfb544db70>: Failed to establish a new connection: [Errno 111] Connection refused

instead of:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/LibreQoS2/v1.2/graphBandwidth.py", line 418, in <module>
    refreshBandwidthGraphs()
  File "/root/LibreQoS2/v1.2/graphBandwidth.py", line 392, in refreshBandwidthGraphs
    write_api.write(bucket=influxDBBucket, record=queriesToSend)
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/client/write_api.py", line 365, in write
    results = list(map(write_payload, payloads.items()))
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/client/write_api.py", line 363, in write_payload
    return self._post_write(_async_req, bucket, org, final_string, payload[0])
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/client/write_api.py", line 474, in _post_write
    return self._write_service.post_write(org=org, bucket=bucket, body=body, precision=precision,
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/service/write_service.py", line 62, in post_write
    (data) = self.post_write_with_http_info(org, bucket, body, **kwargs)  # noqa: E501
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/service/write_service.py", line 92, in post_write_with_http_info
    return self.api_client.call_api(
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
    return self.__call_api(resource_path, method,
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
    response_data = self.request(
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/_sync/api_client.py", line 388, in request
    return self.rest_client.POST(url,
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/_sync/rest.py", line 307, in POST
    return self.request("POST", url,
  File "/usr/local/lib/python3.10/dist-packages/influxdb_client/_sync/rest.py", line 216, in request
    r = self.pool_manager.request(
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 78, in request
    return self.request_encode_body(
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 170, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python3/dist-packages/urllib3/poolmanager.py", line 375, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 507, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.10/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/lib/python3.10/http/client.py", line 975, in send
    self.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fdfb544db70>: Failed to establish a new connection: [Errno 111] Connection refused

dtaht · 2022-11-13T17:06:14Z

I'm punting this to 1.4. I would like to be able to do detailed offline analysis via classic postgres SQL and related tooling myself, and SNMP is useful, as is xabbix, but...

interduo · 2022-11-14T07:21:26Z

The main thing is that we should go out with influx and use external tools for big instances for CPU savings.

interduo · 2022-11-23T18:44:41Z

We generate ${STATS_FILE} with conntent:
Each line of the ${STATS_FILE} must contain 4 whitespace delimited entries:
<hostname> <key> <timestamp> <value>

(every stats read from counters need to just add lines and not replace whole file - If zabbix would be unavaiable then the file will have all missing values from stats and there we would not have a holes in LibreQoS generated graphs. After error code 0 in ealier command we could delete this file.)

The example of zabbix integration could be:

zabbix_sender -z ${ZABBIX_SERVER} -T -i ${STATS_FILE}
if [ $? -eq 0 ]
then 
  rm ${STATS_FILE}
fi

rchac · 2024-05-30T18:43:38Z

Current Solution: LTS

rchac self-assigned this Sep 22, 2022

interduo changed the title ~~Allow to open graphs by CRM~~ Allow data collecting from LibreQoS into monitoring tools Sep 26, 2022

interduo mentioned this issue Sep 26, 2022

Monitoring LibreQOS #6

Closed

dtaht added this to the v1.4 milestone Nov 13, 2022

dtaht modified the milestones: v1.4, v1.5 Jan 12, 2023

rchac closed this as completed May 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow data collecting from LibreQoS into monitoring tools #75

Allow data collecting from LibreQoS into monitoring tools #75

interduo commented Sep 9, 2022

rchac commented Sep 9, 2022

interduo commented Sep 9, 2022 •

edited

Loading

rchac commented Sep 9, 2022

interduo commented Sep 11, 2022 •

edited

Loading

rchac commented Sep 11, 2022 •

edited

Loading

interduo commented Sep 11, 2022 •

edited

Loading

rchac commented Sep 11, 2022

interduo commented Sep 13, 2022

interduo commented Sep 22, 2022 •

edited

Loading

interduo commented Sep 26, 2022

interduo commented Sep 26, 2022

dtaht commented Nov 13, 2022

interduo commented Nov 14, 2022

interduo commented Nov 23, 2022 •

edited

Loading

rchac commented May 30, 2024

Allow data collecting from LibreQoS into monitoring tools #75

Allow data collecting from LibreQoS into monitoring tools #75

Comments

interduo commented Sep 9, 2022

rchac commented Sep 9, 2022

interduo commented Sep 9, 2022 • edited Loading

rchac commented Sep 9, 2022

interduo commented Sep 11, 2022 • edited Loading

rchac commented Sep 11, 2022 • edited Loading

interduo commented Sep 11, 2022 • edited Loading

rchac commented Sep 11, 2022

interduo commented Sep 13, 2022

interduo commented Sep 22, 2022 • edited Loading

interduo commented Sep 26, 2022

interduo commented Sep 26, 2022

dtaht commented Nov 13, 2022

interduo commented Nov 14, 2022

interduo commented Nov 23, 2022 • edited Loading

rchac commented May 30, 2024

interduo commented Sep 9, 2022 •

edited

Loading

interduo commented Sep 11, 2022 •

edited

Loading

rchac commented Sep 11, 2022 •

edited

Loading

interduo commented Sep 11, 2022 •

edited

Loading

interduo commented Sep 22, 2022 •

edited

Loading

interduo commented Nov 23, 2022 •

edited

Loading