logswan

Fast Web log analyzer using probabilistic data structures
Log | Files | Refs | README | LICENSE

logswan.1 (2546B)

      1 .\"
      2 .\" Copyright (c) 2015-2022, Frederic Cambus
      3 .\" All rights reserved.
      4 .\"
      5 .\" Redistribution and use in source and binary forms, with or without
      6 .\" modification, are permitted provided that the following conditions are met:
      7 .\"
      8 .\"   * Redistributions of source code must retain the above copyright
      9 .\"     notice, this list of conditions and the following disclaimer.
     10 .\"
     11 .\"   * Redistributions in binary form must reproduce the above copyright
     12 .\"     notice, this list of conditions and the following disclaimer in the
     13 .\"     documentation and/or other materials provided with the distribution.
     14 .\"
     15 .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
     16 .\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
     17 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
     18 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS
     19 .\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
     20 .\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
     21 .\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
     22 .\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
     23 .\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
     24 .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
     25 .\" POSSIBILITY OF SUCH DAMAGE.
     26 .\"
     27 .Dd $Mdocdate: May 6 2022 $
     28 .Dt LOGSWAN 1
     29 .Os
     30 .Sh NAME
     31 .Nm logswan
     32 .Nd fast Web log analyzer using probabilistic data structures
     33 .Sh SYNOPSIS
     34 .Nm
     35 .Op Fl ghv
     36 .Op Fl d Ar db
     37 .Ar logfile
     38 .Sh DESCRIPTION
     39 .Nm
     40 is a fast Web log analyzer using probabilistic data structures.
     41 It is targeted at very large log files, typically APIs logs.
     42 It has constant memory usage regardless of the log file size, and takes
     43 approximatively 4MB of RAM.
     44 .Pp
     45 Unique visitors counting is performed using two HyperLogLog counters (one for
     46 IPv4, and another one for IPv6), providing a relative accuracy of 0.10%.
     47 .Pp
     48 If
     49 .Ar file
     50 is a single dash (`-'),
     51 .Nm
     52 reads from the standard input.
     53 .Pp
     54 The options are as follows:
     55 .Bl -tag -width Ds
     56 .It Fl d Ar db
     57 Specify path to a GeoIP database.
     58 .It Fl g
     59 Enable GeoIP lookups.
     60 .It Fl h
     61 Display usage.
     62 .It Fl v
     63 Display version.
     64 .El
     65 .Sh EXAMPLES
     66 The following script can be used to process all log files in the current
     67 directory and save the output in a file:
     68 .Bd -literal -offset indent
     69 #!/bin/sh
     70 for file in $(ls *.log)
     71 do
     72 	logswan $file > $file.json
     73 done
     74 exit 0
     75 .Ed
     76 .Sh AUTHORS
     77 .Nm
     78 was written by
     79 .An Frederic Cambus .