logswan

Fast Web log analyzer using probabilistic data structures
Log | Files | Refs | README | LICENSE

ChangeLog (10505B)


      1 Logswan 2.1.13 (2022-07-28)
      2 
      3 - Remove useless backslashes in usage()
      4 - Add entries with IPv6 remote hosts to exercise the IPv6 code paths
      5 - Add an invalid.log log file with invalid lines, for testing purposes
      6 - Add a few more test cases in CMakeLists.txt
      7 - Also generate data for IPv6 addresses in the MMDB test database
      8 - Regenerate the example MMDB database and JSON example output
      9 
     10 
     11 
     12 Logswan 2.1.12 (2021-12-02)
     13 
     14 - Fix a use-after-free (read) triggered by strcmp(3) calls.
     15 
     16   The parse_request() function didn't zero out the parsed_request struct
     17   between each call. Since the parsing loop was switched to using getline(3)
     18   instead of a fixed size buffer to process log lines, it could reference
     19   already freed memory in certain cases.
     20 
     21   Thanks to Brian Carpenter (@geeknik) for finding and reporting the issue.
     22 
     23 
     24 
     25 Logswan 2.1.11 (2021-11-16)
     26 
     27 - Bump CMake minimum required version to 2.8.12
     28 - Add SPDX short license identifier in source files
     29 - Use CMAKE_C_STANDARD instead of hardcoding compiler flags
     30 - Switch to using getline(3) instead of fgets(3)
     31 - Call hll_init() after doing getopt() processing
     32 
     33 
     34 
     35 Logswan 2.1.10 (2021-02-15)
     36 
     37 - Only call MMDB_close if the GeoIP option was enabled. This fixes a
     38   crash on program exit on OpenBSD when running without the GeoIP
     39   option enabled.
     40 
     41 
     42 
     43 Logswan 2.1.9 (2021-02-15)
     44 
     45 - Stop forcing FORTIFY_SOURCE=2, it should be package builders decision
     46 - Add link to Homebrew package in the README
     47 - Remove unused countryId variable
     48 - Remove dead increments for argc and argv
     49 - Rename variables to get rid of all camelCase occurrences
     50 - Remove the measuring Logswan memory usage section from the README
     51 - Try to harmonize usage information everywhere
     52 - Rename the displayUsage() function to usage()
     53 - Rename all the parse*() functions to use snake_case
     54 - Get rid of global variables, move all declarations to main()
     55 
     56 
     57 
     58 Logswan 2.1.8 (2020-10-19)
     59 
     60 - Make displayUsage() static
     61 - Removing some leading whitespace characters
     62 - Constify methods and protocols names array
     63 - Constify countries and continents names and ID arrays
     64 - Enable FORTIFY_SOURCE level 2
     65 - Call memset() to set all entry_data struct fields to zero
     66 
     67 
     68 
     69 Logswan 2.1.7 (2020-09-17)
     70 
     71 - Add a Perl program to generate an example MMDB database for testing
     72 - Add a new test case to exercise the IP geolocation codepaths
     73 - Add support for seccomp on arm
     74 - Add missing test for __NR_mmap, the mmap syscall doesn't exist on arm
     75 
     76 
     77 
     78 Logswan 2.1.6 (2020-07-03)
     79 
     80 - Validate architectures for seccomp
     81 - Minor code shuffling, for consistency with other codebases
     82 - Add seccomp support on i386, tested on glibc and musl
     83 - Remove entry for "Netherlands Antilles" in country table
     84 - Swaziland was officially renamed to Eswatini in 2018
     85 - Macedonia was officially renamed to North Macedonia in February 2019
     86 - Rename Vatican City to Vatican to be consistent with GeoNames naming
     87 - Rename Aland to Aland Islands to be consistent with GeoNames naming
     88 
     89 
     90 
     91 Logswan 2.1.5 (2020-06-12)
     92 
     93 - Switch the GEOIP2DIR example to point to '/var/db/dbip'
     94 - Use CLOCK_REALTIME if CLOCK_MONOTONIC is not available
     95 - Remove unneeded <sys/select.h> and <sys/types.h> includes
     96 - Add missing <stddef.h> include
     97 - Remove useless _POSIX_SOURCE define
     98 - Remove useless _POSIX_C_SOURCE 199309L define
     99 - Define _GNU_SOURCE in CMakeLists.txt to avoid cluttering source files
    100 - No need to set HAVE_SECCOMP to 1, defining it is enough
    101 - Remove some unneeded seccomp related includes
    102 
    103 
    104 
    105 Logswan 2.1.4 (2020-03-27)
    106 
    107 - Move GEOIP2DB definition to CMakeLists.txt, allowing build time
    108   customization
    109 - Display database path name in error message
    110 - Remove direct link to GeoLite2 databases
    111 - Add dependencies installation instruction for Fedora
    112 - Change GEOIP2DB default value to "dbip-country-lite.mmdb"
    113 - Change GEOIP2DIR default value to "${CMAKE_INSTALL_PREFIX}/share/dbip"
    114 - Mention that DB-IP IP to Country Lite database is now the recommended option
    115 
    116 
    117 
    118 Logswan 2.1.3 (2020-01-17)
    119 
    120 - Add a new test target, to test log processing
    121 - Move printed statistics after the actual output
    122 - Use OpenBSD style(9) for function prototypes and declarations
    123 - Remove seccomp mention in README as it is currently disabled by default
    124 
    125 
    126 
    127 Logswan 2.1.2 (2019-11-19)
    128 
    129 - Add ENABLE_SECCOMP build option, to allow building seccomp support
    130   conditionally
    131 - Disable seccomp by default, it needs more testing on non !amd64 platforms
    132 - Use ${CMAKE_INSTALL_BINDIR} instead of hardcoding 'bin'
    133 
    134 
    135 
    136 Logswan 2.1.1 (2019-10-30)
    137 
    138 - Check if system has seccomp in CMakeLists.txt
    139 - Use the HAVE_SECCOMP macro to check whether or not to enable seccomp
    140 - Define and use a GEOIP2DB macro to specify GeoLite2 database name
    141 - Add a switch (-d) to allow specifying path to a GeoIP2 database file
    142 - Define and use a LOGSWAN_SYSCALL_ALLOW macro to make code more readable
    143 - Adding missing #include guard in seccomp.h header file
    144 - Use __NR_ instead of SYS_ prefix in LOGSWAN_SYSCALL_ALLOW
    145 - Fix the build on aarch64 Linux, where the open() syscall does not exist
    146 - Add error checking for both prctl() calls
    147 
    148 
    149 
    150 Logswan 2.1.0 (2019-10-23)
    151 
    152 - Add FALLTHROUGH comments where appropriate
    153 - Add support for parsing HTTP/3 requests
    154 - Add initial seccomp support on Linux, tested on musl and glibc systems
    155 
    156 
    157 
    158 Logswan 2.0.4 (2019-08-16)
    159 
    160 - Adding #include guard in compat header file
    161 - Add an example log file and regenerate output example
    162 - Add dependencies installation instructions for NetBSD and FreeBSD
    163 - Add final dots for options descriptions
    164 - Add final dot when printing results summary
    165 - Use EXIT_SUCCESS and EXIT_FAILURE macros for return values
    166 - Add a trailing newline when printing JSON output
    167 
    168 
    169 
    170 Logswan 2.0.3 (2018-10-15)
    171 
    172 - Use -std=c11, Logswan requires a C11 compiler for libmaxminddb
    173 - Enable support for parsing HTTP/2.0 requests, for real this time
    174 - Revert back to using INT64_MAX for strtonum() maxval, as maxval is long long
    175   and using UINT64_MAX caused bandwidth parsing to always fail (Thanks James Loh
    176   for reporting the issue)
    177 - Move maps of countries and continents to separate files
    178 
    179 
    180 
    181 Logswan 2.0.2 (2018-08-05)
    182 
    183 - Use UINT64_MAX for strtonum() maxval
    184 - Add missing headers and reorder includes
    185 - Add support for HTTP/2.0
    186 
    187 
    188 
    189 Logswan 2.0.1 (2018-06-27)
    190 
    191 - Do not use -Werror by default
    192 - Do not always call inet_pton two times per log line, this speeds things up
    193 - Use bool types for isIPv4 and isIPv6
    194 - Use the monotonic clock to determine runtime
    195 - Move conditional includes for 'pledge' and 'strtonum' in compat.h
    196 
    197 
    198 
    199 Logswan 2.0.0 (2018-03-16)
    200 
    201 - Use type off_t for results struct member fileSize
    202 - Reflect OpenBSD's pledge() changes
    203 - Switch to using libmaxminddb and GeoIP2/GeoLite2 databases
    204 - Add Antarctica to the list of continents
    205 
    206 
    207 
    208 Logswan 1.07 (2017-02-14)
    209 
    210 - Harmonize arrays names
    211 - Remove array of months, it's currently unused and will likely remain so
    212 - Simplify internal JSON array and object names
    213 - Use OpenBSD style(9) for function prototypes and declarations
    214 - Revert back to using strtok, at least for now
    215 - Do not use EXIT_SUCCESS and EXIT_FAILURE macros anymore
    216 - Fix implicit function declaration error on NetBSD (Thanks Maya Rashish)
    217 - Remove now useless variables initialization and unnecessary includes
    218 - Do not add an extra new line when displaying usage or version
    219 
    220 
    221 
    222 Logswan 1.06 (2016-12-17)
    223 
    224 - Relicensed under the BSD 2-Clause license
    225 - Use strtok_r instead of strtok to tokenize lines
    226 - Do not attempt to increment countries and continents arrays if there
    227   is no GeoIP database loaded
    228 - Harmonize variable names for the GeoIP databases
    229 - GeoIP lookups are now disabled by default (add a -g switch to enable)
    230 - Use fstat on open file descriptor instead of using stat before opening
    231   the input file
    232 - Count the log line as invalid if parsedLine.remoteHost is NULL
    233 - Pass results structure by reference, not by value
    234 - Initialize some uninitialized variables
    235 - Headers cleanup
    236 
    237 
    238 
    239 Logswan 1.05 (2016-02-25)
    240 
    241 - Documentation update (notes on measuring Logswan memory usage)
    242 - Add additional include directories for compat functions + dependencies
    243   to avoid using relative path in includes
    244 - Check that *lineBuffer is not NUL before attempting to parse log line
    245 - Perform GeoIP lookup and HLL add in the same if block
    246 - Increment IPv4 and IPv6 hits counters individually and conditionally
    247 - Use CMake to check if the system has OpenBSD's pledge available and
    248   link pledge conditionally using a null implementation when compiled
    249   on non OpenBSD systems
    250 - Adding an array of months, for the upcoming split log functionalities
    251 
    252 
    253 
    254 Logswan 1.04 (2016-01-10)
    255 
    256 - Moving global variables into main
    257 - Using 'size_t' instead of 'int' for array indexes in for loops
    258 - Using 'uint32_t' for all non 'uint64_t' integers
    259 - Do not increment hits and processed lines counter for each parsed line,
    260   compute total only once when everything is parsed
    261 - Setting 'CMAKE_BUILD_TYPE' to 'Release' and formatting fixes
    262 - Sanitize CMake script to build under NetBSD (Thanks Kamil Rytarowski)
    263 - Initializing some uninitialized variables
    264 - Renaming 'DATADIR' variables to 'GEOIPDIR'
    265 
    266 
    267 
    268 Logswan 1.03 (2016-01-01)
    269 
    270 - Remove header display and do not print name of processed file
    271 - Print results to stderr instead of stdout
    272 - Output JSON data to stdout instead of creating a new file
    273 - Define GeoIP databases path in CMakeLists.txt
    274 - Adding log file name in the JSON output
    275 - Removing some hardcoded values and replacing them with constants
    276   defined in config.h
    277 - Breaking the loop when a match is found in the request parser
    278 - Using enumeration constants instead of macros
    279 - Process GeoIP continent information
    280 - Re-ordering protocols and methods with more common occurrences on top of
    281   the list, allowing to break earlier when iterating through the array
    282 - Adding support for reading logs from standard input
    283 - Renaming 'definitions' files to 'config'
    284 - Increasing countries array size, as an attempt to be future-proof
    285 - Initial support for using pledge() on OpenBSD
    286 - Documentation updates (HLL precision, Features list, GeoIP databases)
    287 - Updated JSON output example
    288 - Added a manual page
    289 
    290 
    291 
    292 Logswan 1.02 (2015-11-02)
    293 
    294 - Renaming 'resource' variable to 'request' in the 'logLine' struct
    295 - Do not attempt to parse empty date tokens
    296 - Do not attempt to parse empty request tokens (Thanks Brian Carpenter for
    297 reporting the issue)
    298 
    299 
    300 
    301 Logswan 1.01 (2015-10-01)
    302 
    303 - Documentation updates
    304 - Fixing segfault when request data is empty or malformed (Thanks Jonathan
    305   Armani for reporting and proposing a fix)
    306 
    307 
    308 
    309 Logswan 1.00 (2015-09-28)
    310 
    311 - Initial release