logswan

Fast Web log analyzer using probabilistic data structures
Log | Files | Refs | README | LICENSE

commit 84a6b2fa9084c814707ac3742378594708006174
parent 022bdddcd3c870143dac3b4cbd9d2b87ab7598e5
Author: Frederic Cambus <fred@statdns.com>
Date:   Tue,  6 Dec 2016 20:42:14 +0100

Reformat README, AUTHORS files and man page to wrap long lines and remove whitespace before colons

Diffstat:
MAUTHORS | 6+++---
MREADME.md | 74+++++++++++++++++++++++++++++++++++++++++++++-----------------------------
Mlogswan.1 | 4+++-
3 files changed, 51 insertions(+), 33 deletions(-)

diff --git a/AUTHORS b/AUTHORS @@ -1,6 +1,6 @@ -Logswan is developed by : +Logswan is developed by: Frederic Cambus <fred AT statdns DOT com> -Site : http://www.cambus.net -Twitter : @fcambus +Site: http://www.cambus.net +Twitter: @fcambus diff --git a/README.md b/README.md @@ -28,21 +28,31 @@ ``` # Logswan [![Build Status][1]][2] -Logswan is a fast Web log analyzer using probabilistic data structures. It is targeted at very large log files, typically APIs logs. It has constant memory usage regardless of the log file size, and takes approximatively 4MB of RAM. +Logswan is a fast Web log analyzer using probabilistic data structures. It is +targeted at very large log files, typically APIs logs. It has constant memory +usage regardless of the log file size, and takes approximatively 4MB of RAM. -Unique visitors counting is performed using two HyperLogLog counters (one for IPv4, and another one for IPv6), providing a relative accuracy of 0.10%. String representations of IP addresses are used and preferred as they offer better precision. +Unique visitors counting is performed using two HyperLogLog counters (one for +IPv4, and another one for IPv6), providing a relative accuracy of 0.10%. +String representations of IP addresses are used and preferred as they offer +better precision. -Project design goals include : speed, memory-usage efficiency, and keeping the code as simple as possible. +Project design goals include: speed, memory-usage efficiency, and keeping the +code as simple as possible. -Logswan is **opinionated software** : +Logswan is **opinionated software**: -- It only supports the Common Log Format, in order to keep the parsing code simple. It can of course process the Combined Log Format as well (referer and user agent fields will be discarded) -- It does not split results per day, but log files can be split prior to being processed -- Input file size and bandwidth usage are reported in bytes, there are no plans to format or round them +- It only supports the Common Log Format, in order to keep the parsing code + simple. It can of course process the Combined Log Format as well (referer + and user agent fields will be discarded) +- It does not split results per day, but log files can be split prior to + being processed +- Input file size and bandwidth usage are reported in bytes, there are no + plans to format or round them ## Features -Currently implemented features : +Currently implemented features: - Counting bandwidth used - Counting number of processed lines / invalid lines @@ -56,26 +66,28 @@ Currently implemented features : ## Dependencies -Logswan uses the `CMake` build system and requires `GeoIP` and `Jansson` libraries and header files. +Logswan uses the `CMake` build system and requires `GeoIP` and `Jansson` +libraries and header files. ## Installing dependencies -- OpenBSD : `pkg_add -r cmake GeoIP jansson` -- NetBSD : `pkgin install cmake GeoIP jansson` -- Mac OS X : `brew install cmake geoip jansson` -- Alpine Linux : `apk add cmake gcc make musl-dev geoip-dev jansson-dev` -- Debian / Ubuntu : `apt-get install build-essential cmake libgeoip-dev libjansson-dev` +- OpenBSD: `pkg_add -r cmake GeoIP jansson` +- NetBSD: `pkgin install cmake GeoIP jansson` +- Mac OS X: `brew install cmake geoip jansson` +- Alpine Linux: `apk add cmake gcc make musl-dev geoip-dev jansson-dev` +- Debian / Ubuntu: `apt-get install build-essential cmake libgeoip-dev libjansson-dev` ## Compiling cmake . make -Logswan has been sucessfully compiled and tested on Mac OS X, OpenBSD, NetBSD, and Linux with both Clang and GCC. +Logswan has been sucessfully compiled and tested on Mac OS X, OpenBSD, NetBSD, +and Linux with both Clang and GCC. ## Installation -Logswan packages are available for : +Logswan packages are available for: ### OpenBSD @@ -87,60 +99,64 @@ Logswan packages are available for : ### GeoIP databases -By default, Logswan looks for GeoIP databases in `${CMAKE_INSTALL_PREFIX}/share/GeoIP`, which points to `/usr/local/share/GeoIP` by default. +Logswan looks for GeoIP databases in `${CMAKE_INSTALL_PREFIX}/share/GeoIP` by +default, which points to `/usr/local/share/GeoIP`. -A custom directory can be set using the `GEOIPDIR` variable when invoking CMake : +A custom directory can be set using the `GEOIPDIR` variable when invoking +CMake: cmake -DGEOIPDIR=/var/db/GeoIP . -The free GeoLite databases can be downloaded here : http://dev.maxmind.com/geoip/legacy/geolite/ +The free GeoLite databases from MaxMind can be downloaded [here][3]. -#### For IPv4 support only : +#### For IPv4 support only https://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz -#### For IPv4 and IPv6 support : +#### For IPv4 and IPv6 support https://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz https://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz -## Usage +## Usage logswan [-ghv] file If file is a single dash (`-'), logswan reads from the standard input. -Options are : +Options are: -g Enable GeoIP lookups -h Display usage -v Display version -Logswan outputs JSON data to **stdout**. +Logswan outputs JSON data to **stdout**. ## Measuring Logswan memory usage -Heap profiling can be done using valgrind, as follow : +Heap profiling can be done using valgrind, as follow: valgrind --tool=massif logswan access.log ms_print massif.out.* ## License -Logswan is released under the BSD 2-Clause license. See `LICENSE` file for details. +Logswan is released under the BSD 2-Clause license. See `LICENSE` file for +details. ## Author Logswan is developed by Frederic Cambus -- Site : http://www.cambus.net +- Site: http://www.cambus.net - Twitter: http://twitter.com/fcambus ## Resources -Project Homepage : http://www.logswan.org +Project homepage: http://www.logswan.org -GitHub : https://github.com/fcambus/logswan +GitHub: https://github.com/fcambus/logswan [1]: https://travis-ci.org/fcambus/logswan.png?branch=master [2]: https://travis-ci.org/fcambus/logswan +[3]: http://dev.maxmind.com/geoip/legacy/geolite/ diff --git a/logswan.1 b/logswan.1 @@ -36,7 +36,9 @@ .Ar file .Sh DESCRIPTION .Nm -is a fast Web log analyzer using probabilistic data structures. It is targeted at very large log files, typically APIs logs. It has constant memory usage regardless of the log file size, and takes approximatively 4MB of RAM. +is a fast Web log analyzer using probabilistic data structures. It is targeted +at very large log files, typically APIs logs. It has constant memory usage +regardless of the log file size, and takes approximatively 4MB of RAM. .Pp If .Ar file