Compare commits

...

60 commits

Author SHA1 Message Date
Daniel Lemire
bc93aee338
Merge pull request #80 from BYVoid/bench
Add benchmark binary to Bazel BUILD.
2024-10-31 11:14:09 -04:00
Carbo Kuo
f5096c7a94 Add benchmark binary to Bazel BUILD.
To build it, run:
bazel build --compilation_mode=opt //:benchmark

To run the benchmark, run:
bazel-bin/benchmark benchmarks/data/canada.txt

Result from my computer:

read 111126 lines

=== trial 1 ===
fast_double_parser  1315.70 MB/s
strtod         722.10 MB/s
abslfromch     810.26 MB/s
absl           799.89 MB/s
double-conv    396.05 MB/s

=== trial 2 ===
fast_double_parser  1416.58 MB/s
strtod         751.43 MB/s
abslfromch     841.83 MB/s
absl           838.71 MB/s
double-conv    415.33 MB/s
2024-10-31 10:39:29 -04:00
Daniel Lemire
22ac46158c
Merge pull request #79 from BYVoid/bazel-build
Add a simple Bazel module and build file.
2024-10-31 09:48:55 -04:00
Carbo Kuo
3aaddfe336 Add a simple Bazel module and build file. 2024-10-31 08:44:43 -04:00
Daniel Lemire
252029ddac
Update README.md 2024-05-07 10:01:17 -04:00
Daniel Lemire
d971fa76ee update 2024-05-07 09:59:45 -04:00
Daniel Lemire
305e6c8170 better cmake 2024-03-18 10:17:36 -04:00
Daniel Lemire
d61b589f65 tuning. 2024-03-18 10:03:14 -04:00
Daniel Lemire
b92d89b2c4 updating 2024-03-18 10:01:21 -04:00
Daniel Lemire
ca05d13e26
Merge pull request #77 from Osyotr/cmake-exports
Export cmake config and add option to build unit tests
2024-03-18 09:03:29 -04:00
Osyotr
2a3319d698 Export cmake config and add option to build unit tests 2024-03-17 22:45:44 +03:00
Osyotr
468ee515f4 Add visual studio files to gitignore 2024-03-17 22:18:48 +03:00
Daniel Lemire
172f42bccf
Merge pull request #76 from barracuda156/apple
CMakeLists: do not use Linux linker flag on macOS
2024-03-12 13:15:34 -04:00
Sergey Fedorov
da4ba02ea7
CMakeLists: do not use Linux linker flag on macOS 2024-03-10 01:17:32 +07:00
Daniel Lemire
4f5e530b02
Update README.md 2023-12-26 16:06:21 -05:00
Daniel Lemire
15f93a61bb
Update README.md 2023-12-26 16:05:15 -05:00
Daniel Lemire
3489ead54c
Merge pull request #75 from BebeSparkelSparkel/master
Deprecation notice at top
2023-12-26 16:04:07 -05:00
William Rusnack
b94e499b22
Deprecation notice at top 2023-12-26 11:16:07 -05:00
Daniel Lemire
d2e8f223e7
Merge pull request #74 from DavidKorczynski/clusterfuzzlite-integration
Add ClusterFuzzLite integration
2023-12-21 09:48:50 -05:00
David Korczynski
06bc1ea3ed Add ClusterFuzzLite integration
Signed-off-by: David Korczynski <david@adalogics.com>
2023-12-21 03:48:47 -08:00
Daniel Lemire
38c6d6da54 removing old ci 2023-12-06 21:15:13 -05:00
Daniel Lemire
56a813c54f added test. 2023-12-06 21:14:43 -05:00
Daniel Lemire
cee3272a5c
Update README.md 2023-12-02 18:54:05 -05:00
Daniel Lemire
07d9189a8f
Merge pull request #72 from striezel-stash/update-actions-setup-cmake
Update jwlawson/actions-setup-cmake in GitHub Actions to v1.13
2023-01-10 10:24:35 -05:00
Dirk Stolle
8a55baef01 Update jwlawson/actions-setup-cmake in GitHub Actions to v1.13 2023-01-09 22:56:33 +01:00
Daniel Lemire
d039d6fe5e
Merge pull request #71 from nxtn-staged/patch-1
Use _BitScanReverse on Win32
2023-01-03 09:50:49 -05:00
nxtn-staged
5a8c7606df
Use _BitScanReverse on Win32 2022-12-27 11:23:19 +08:00
Daniel Lemire
efec03532e
Merge pull request #67 from lgtm-migrator/codeql
Add CodeQL workflow for GitHub code scanning
2022-12-08 10:20:27 -05:00
Daniel Lemire
027fb3abd3
Merge pull request #69 from striezel-stash/action-update
Update actions/checkout in GitHub Actions to v3
2022-12-08 10:20:15 -05:00
Daniel Lemire
749ba90831
Merge pull request #70 from zheddie/master
Supporting IBM i/PASE env.
2022-12-08 10:20:02 -05:00
GavinZhang
0e2a13868f Refine code according to the comments from reviewer. 2022-12-08 17:10:53 +08:00
GavinZhang
2df5ff039b Supporting IBM i/PASE env. 2022-12-07 17:09:20 +08:00
Daniel Lemire
7aebf2f747
Merge pull request #68 from striezel-stash/readme-typo
Fix a typo in README.md
2022-11-15 18:25:41 -05:00
Dirk Stolle
1427971755 Update actions/checkout in GitHub Actions to v3 2022-11-16 00:19:07 +01:00
Dirk Stolle
4a73a889f0 Fix a typo in README.md 2022-11-16 00:13:34 +01:00
LGTM Migrator
e8c1bf481e
Add CodeQL workflow for GitHub code scanning 2022-11-10 21:03:54 +00:00
Daniel Lemire
66657d1a48
Update README.md 2022-11-03 19:15:38 -04:00
Daniel Lemire
357eec4e61 Adding test. 2022-10-02 10:59:26 -04:00
Daniel Lemire
267b118aec
Update README.md 2022-07-24 12:44:28 -04:00
Daniel Lemire
b823bbc8a0
Merge pull request #64 from lemire/dlemire/issue63
Adding test for issue 63.
2022-07-09 13:47:17 -04:00
Daniel Lemire
c3b92a47c7 Adding test for issue 63. 2022-07-09 13:39:27 -04:00
Daniel Lemire
50a2ccb553
Update README.md 2022-05-19 12:32:19 -04:00
Daniel Lemire
4ca9eb3221
Merge pull request #62 from lemire/dlemire/upgradingtovs17
Upgrading tests to latest visual studio
2022-05-19 12:31:07 -04:00
Daniel Lemire
3b296fc600 Fixing legacy Win32 support 2022-05-19 12:25:13 -04:00
Daniel Lemire
182c61b1ce Removing mingw 2022-05-19 09:54:31 -04:00
Daniel Lemire
8ed6508c0d Upgrading to latest visual studio 2022-05-19 09:52:42 -04:00
Daniel Lemire
36f2c5ef59
Merge pull request #61 from ruggiero/master
Fix x32 variable
2022-05-19 09:46:04 -04:00
Rafael Ruggiero
b3c7fce960
Fix x32 variable 2022-05-19 02:30:15 -03:00
Daniel Lemire
ffe3619894
Update README.md 2022-03-07 09:47:50 -05:00
Daniel Lemire
111ad80417
Update README.md 2021-10-23 18:04:54 -04:00
Daniel Lemire
30302eac8c
Update README.md 2021-07-02 14:18:26 -04:00
Daniel Lemire
715cf49f1a
Merge pull request #56 from lemire/issue49
Patch for legacy systems
2021-05-18 14:40:50 -04:00
Daniel Lemire
4d1ada99c2 Removing para 2021-05-18 14:33:25 -04:00
Daniel Lemire
70f4b8fac1 Patch. 2021-05-18 14:31:23 -04:00
Daniel Lemire
2cc127528c
Update CMakeLists.txt 2021-05-18 13:54:18 -04:00
Daniel Lemire
fa50da4ffb
Update README.md 2021-04-23 09:23:01 -04:00
Daniel Lemire
e4f6319bfa
Merge pull request #54 from wojdyr/b1
move arrays from file scope to compute_float_64()
2021-04-01 13:07:37 -04:00
Marcin Wojdyr
b40d08a245 move arrays from file scope to compute_float_64()
It avoids duplication of the arrays in each compilation unit,
saving about (N-1)*10Kb in the binary size.
2021-04-01 18:38:59 +02:00
Daniel Lemire
c0b367096a
Merge pull request #52 from wojdyr/patch-1
README.md: fix parse_number return type
2021-03-31 17:48:05 -04:00
Marcin Wojdyr
cf0f3daeaa
README.md: fix parse_number return type 2021-03-31 23:29:24 +02:00
29 changed files with 380 additions and 283 deletions

View file

@ -22,9 +22,9 @@ build_script:
- mkdir build
- cd build
- cmake --version
- cmake %CMAKE_ARGS% --parallel ..
- cmake %CMAKE_ARGS% ..
- cmake ..
- cmake --build . --config %Configuration% --verbose --parallel
- cmake --build . --config %Configuration% --verbose
for:
-
@ -33,7 +33,7 @@ for:
- job_name: VS2019ARM
test_script:
- ctest --output-on-failure -C %Configuration% --verbose %CTEST_ARGS% --parallel
- ctest --output-on-failure -C %Configuration% --verbose %CTEST_ARGS%
clone_folder: c:\projects\fast_double_parser

View file

@ -0,0 +1,6 @@
FROM gcr.io/oss-fuzz-base/base-builder
RUN apt-get update && apt-get install -y make autoconf automake libtool
COPY . $SRC/fast_double_parser
COPY .clusterfuzzlite/build.sh $SRC/build.sh
WORKDIR $SRC/fast_double_parser

View file

@ -0,0 +1,3 @@
# ClusterFuzzLite set up
This folder contains a fuzzing set for [ClusterFuzzLite](https://google.github.io/clusterfuzzlite).

View file

@ -0,0 +1,7 @@
#!/bin/bash -eu
# Copy all fuzzer executable to $OUT/
$CXX $CFLAGS $LIB_FUZZING_ENGINE \
$SRC/fast_double_parser/.clusterfuzzlite/parse_number_fuzzer.cpp \
-o $OUT/parse_number_fuzzer \
-I$SRC/fast_double_parser/include

View file

@ -0,0 +1,11 @@
#include <fast_double_parser.h>
#include <string>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
std::string fuzz_input(reinterpret_cast<const char *>(data), size);
double x;
fast_double_parser::parse_number(fuzz_input.c_str(), &x);
return 0;
}

View file

@ -0,0 +1 @@
language: c++

30
.github/workflows/cflite_pr.yml vendored Normal file
View file

@ -0,0 +1,30 @@
name: ClusterFuzzLite PR fuzzing
on:
workflow_dispatch:
pull_request:
branches: [ main ]
permissions: read-all
jobs:
PR:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sanitizer: [address]
steps:
- name: Build Fuzzers (${{ matrix.sanitizer }})
id: build
uses: google/clusterfuzzlite/actions/build_fuzzers@v1
with:
sanitizer: ${{ matrix.sanitizer }}
language: c++
bad-build-check: false
- name: Run Fuzzers (${{ matrix.sanitizer }})
id: run
uses: google/clusterfuzzlite/actions/run_fuzzers@v1
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
fuzz-seconds: 100
mode: 'code-change'
report-unreproducible-crashes: false
sanitizer: ${{ matrix.sanitizer }}

41
.github/workflows/codeql.yml vendored Normal file
View file

@ -0,0 +1,41 @@
name: "CodeQL"
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
schedule:
- cron: "1 8 * * 3"
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ cpp ]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
queries: +security-and-quality
- name: Autobuild
uses: github/codeql-action/autobuild@v2
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{ matrix.language }}"

View file

@ -1,49 +0,0 @@
name: MinGW32-CI
on: [push, pull_request]
# Important: scoop will either install 32-bit GCC or 64-bit GCC, not both.
# It is important to build static libraries because cmake is not smart enough under Windows/mingw to take care of the path. So
# with a dynamic library, you could get failures due to the fact that the EXE can't find its DLL.
jobs:
ci:
name: windows-gcc
runs-on: windows-2016
env:
CMAKE_GENERATOR: Ninja
CC: gcc
CXX: g++
steps: # To reproduce what is below, start a powershell with administrative rights, using scoop *is* a good idea
- uses: actions/checkout@v2
- uses: actions/cache@v2 # we cache the scoop setup with 32-bit GCC
id: cache
with:
path: |
C:\ProgramData\scoop
key: scoop32 # static key: should be good forever
- name: Setup Windows # This should almost never run if the cache works.
if: steps.cache.outputs.cache-hit != 'true'
shell: powershell
run: |
Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')
scoop install sudo --global
sudo scoop install git --global
sudo scoop install ninja --global
sudo scoop install cmake --global
sudo scoop install gcc --arch 32bit --global
$env:path
Write-Host 'Everything has been installed, you are good!'
- name: Build and Test 32-bit x86
shell: powershell
run: |
$ENV:PATH="C:\ProgramData\scoop\shims;C:\ProgramData\scoop\apps\gcc\current\bin;C:\ProgramData\scoop\apps\ninja\current;$ENV:PATH"
mkdir build32
cd build32
cmake ..
cmake --build . --verbose
ctest -j4 --output-on-failure

View file

@ -1,49 +0,0 @@
name: MinGW64-CI
on: [push, pull_request]
# Important: scoop will either install 32-bit GCC or 64-bit GCC, not both.
# It is important to build static libraries because cmake is not smart enough under Windows/mingw to take care of the path. So
# with a dynamic library, you could get failures due to the fact that the EXE can't find its DLL.
jobs:
ci:
name: windows-gcc
runs-on: windows-2016
env:
CMAKE_GENERATOR: Ninja
CC: gcc
CXX: g++
steps: # To reproduce what is below, start a powershell with administrative rights, using scoop *is* a good idea
- uses: actions/checkout@v2
- uses: actions/cache@v2 # we cache the scoop setup with 64-bit GCC
id: cache
with:
path: |
C:\ProgramData\scoop
key: scoop64 # static key: should be good forever
- name: Setup Windows # This should almost never run if the cache works.
if: steps.cache.outputs.cache-hit != 'true'
shell: powershell
run: |
Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')
scoop install sudo --global
sudo scoop install git --global
sudo scoop install ninja --global
sudo scoop install cmake --global
sudo scoop install gcc --arch 64bit --global
$env:path
Write-Host 'Everything has been installed, you are good!'
- name: Build and Test 64-bit x64
shell: powershell
run: |
$ENV:PATH="C:\ProgramData\scoop\shims;C:\ProgramData\scoop\apps\gcc\current\bin;C:\ProgramData\scoop\apps\ninja\current;$ENV:PATH"
mkdir build64
cd build64
cmake ..
cmake --build . --verbose
ctest -j4 --output-on-failure

View file

@ -29,7 +29,7 @@ jobs:
CMAKE_GENERATOR: Ninja
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: msys2/setup-msys2@v2
with:
update: true

View file

@ -29,7 +29,7 @@ jobs:
CMAKE_GENERATOR: Ninja
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: msys2/setup-msys2@v2
with:
update: true

View file

@ -7,7 +7,7 @@ jobs:
ubuntu-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: start docker
run: |
docker run -w /src -dit --name rhub -v $PWD:/src rhub/rocker-gcc-san

View file

@ -1,16 +1,15 @@
name: Ubuntu 20.04 CI (GCC 9)
name: Ubuntu 22.04
on: [push, pull_request]
jobs:
ubuntu-build:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
strategy:
matrix:
cxx: [g++-12, clang++-14]
steps:
- uses: actions/checkout@v2
- name: Setup cmake
uses: jwlawson/actions-setup-cmake@v1.4
with:
cmake-version: '3.9.x'
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
- name: Use cmake
run: |
mkdir build &&

View file

@ -1,20 +0,0 @@
name: Ubuntu 18.04 CI (GCC 7)
on: [push, pull_request]
jobs:
ubuntu-build:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
- name: Setup cmake
uses: jwlawson/actions-setup-cmake@v1.4
with:
cmake-version: '3.9.x'
- name: Use cmake
run: |
mkdir build &&
cd build &&
cmake -DFAST_DOUBLE_BENCHMARKS=ON .. &&
cmake --build . &&
ctest -j --output-on-failure

View file

@ -1,24 +0,0 @@
name: VS16-CI
on: [push, pull_request]
jobs:
ci:
name: windows-vs16
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: 'Run CMake with VS16'
uses: lukka/run-cmake@v2
with:
cmakeListsOrSettingsJson: CMakeListsTxtAdvanced
cmakeListsTxtPath: '${{ github.workspace }}/CMakeLists.txt'
buildDirectory: "${{ github.workspace }}/../../_temp/windows"
cmakeBuildType: Release
buildWithCMake: true
cmakeGenerator: VS16Win64
buildWithCMakeArgs: --config Release
- name: 'Run CTest'
run: ctest -C Release --output-on-failure
working-directory: "${{ github.workspace }}/../../_temp/windows"

View file

@ -1,25 +0,0 @@
name: VS16-CLANG-CI
on: [push, pull_request]
jobs:
ci:
name: windows-vs16
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: 'Run CMake with VS16'
uses: lukka/run-cmake@v2
with:
cmakeListsOrSettingsJson: CMakeListsTxtAdvanced
cmakeListsTxtPath: '${{ github.workspace }}/CMakeLists.txt'
buildDirectory: "${{ github.workspace }}/../../_temp/windows"
cmakeBuildType: Release
buildWithCMake: true
cmakeGenerator: VS16Win64
cmakeAppendedArgs: -T ClangCL
buildWithCMakeArgs: --config Release
- name: 'Run CTest'
run: ctest -C Release --output-on-failure
working-directory: "${{ github.workspace }}/../../_temp/windows"

View file

@ -1,25 +0,0 @@
name: VS16-Ninja-CI
on: [push, pull_request]
jobs:
ci:
name: windows-vs16
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: 'Run CMake with VS16'
uses: lukka/run-cmake@v2
with:
cmakeListsOrSettingsJson: CMakeListsTxtAdvanced
cmakeListsTxtPath: '${{ github.workspace }}/CMakeLists.txt'
buildDirectory: "${{ github.workspace }}/../../_temp/windows"
cmakeBuildType: Release
buildWithCMake: true
cmakeGenerator: VS16Win64
cmakeAppendedArgs: -G Ninja
buildWithCMakeArgs: --config Release
- name: 'Run CTest'
run: ctest -C Release --output-on-failure
working-directory: "${{ github.workspace }}/../../_temp/windows"

37
.github/workflows/vs17-ci.yml vendored Normal file
View file

@ -0,0 +1,37 @@
name: VS17-CI
on: [push, pull_request]
jobs:
ci:
if: >-
! contains(toJSON(github.event.commits.*.message), '[skip ci]') &&
! contains(toJSON(github.event.commits.*.message), '[skip github]')
name: windows-vs17
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
include:
- {gen: Visual Studio 17 2022, arch: Win32}
- {gen: Visual Studio 17 2022, arch: Win32}
- {gen: Visual Studio 17 2022, arch: x64}
- {gen: Visual Studio 17 2022, arch: x64}
steps:
- name: checkout
uses: actions/checkout@v3
- name: Configure
run: |
cmake -G "${{matrix.gen}}" -A ${{matrix.arch}} -B build
- name: Build Debug
run: cmake --build build --config Debug --verbose
- name: Build Release
run: cmake --build build --config Release --verbose
- name: Run Release tests
run: |
cd build
ctest -C Release --output-on-failure
- name: Run Debug tests
run: |
cd build
ctest -C Debug --output-on-failure

37
.github/workflows/vs17-clang-ci.yml vendored Normal file
View file

@ -0,0 +1,37 @@
name: VS17-CLANG-CI
on: [push, pull_request]
jobs:
ci:
if: >-
! contains(toJSON(github.event.commits.*.message), '[skip ci]') &&
! contains(toJSON(github.event.commits.*.message), '[skip github]')
name: windows-vs17
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
include:
- {gen: Visual Studio 17 2022, arch: Win32}
- {gen: Visual Studio 17 2022, arch: Win32}
- {gen: Visual Studio 17 2022, arch: x64}
- {gen: Visual Studio 17 2022, arch: x64}
steps:
- name: checkout
uses: actions/checkout@v3
- name: Configure
run: |
cmake -G "${{matrix.gen}}" -A ${{matrix.arch}} -T ClangCL -B build
- name: Build Debug
run: cmake --build build --config Debug --verbose
- name: Build Release
run: cmake --build build --config Release --verbose
- name: Run Release tests
run: |
cd build
ctest -C Release --output-on-failure
- name: Run Debug tests
run: |
cd build
ctest -C Debug --output-on-failure

4
.gitignore vendored
View file

@ -44,3 +44,7 @@
/fast_double_parser.cxxflags
/fast_double_parser.files
/fast_double_parser.includes
# Visual Studio
/.vs
/out

22
BUILD.bazel Normal file
View file

@ -0,0 +1,22 @@
cc_library(
name = "fast_double_parser",
hdrs = ["include/fast_double_parser.h"],
strip_include_prefix = "include",
visibility = ["//visibility:public"],
)
cc_test(
name = "unit",
srcs = ["tests/unit.cpp"],
deps = [":fast_double_parser"],
)
cc_binary(
name = "benchmark",
srcs = ["benchmarks/benchmark.cpp"],
deps = [
":fast_double_parser",
"@abseil-cpp//absl/strings",
"@double-conversion",
],
)

View file

@ -1,6 +1,7 @@
cmake_minimum_required(VERSION 3.11)
cmake_policy(SET CMP0048 NEW)
project(fast_double_parser LANGUAGES CXX VERSION 0.0.0.0)
cmake_minimum_required(VERSION 3.9)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
if (NOT CMAKE_BUILD_TYPE)
@ -19,53 +20,82 @@ set(benchmark_src benchmarks/benchmark.cpp)
add_library(fast_double_parser INTERFACE)
target_include_directories(fast_double_parser INTERFACE include/)
target_include_directories(fast_double_parser
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}/include>
$<INSTALL_INTERFACE:include>
)
include(GNUInstallDirs)
install(FILES ${headers} DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}")
install(TARGETS fast_double_parser EXPORT fast_double_parser-targets)
install(
EXPORT fast_double_parser-targets
DESTINATION "share/fast_double_parser"
NAMESPACE fast_double_parser::
)
add_executable(unit ${unit_src} ${bogus_src} ${rebogus_src})
if(FAST_DOUBLE_PARSER_SANITIZE)
target_compile_options(unit PUBLIC -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=all)
target_link_options(unit PUBLIC -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=all)
# Ubuntu bug for GCC 5.0+ (safe for all versions)
if (CMAKE_COMPILER_IS_GNUCC)
target_link_libraries(unit PUBLIC -fuse-ld=gold)
endif()
include(CMakePackageConfigHelpers)
configure_package_config_file(
"${CMAKE_CURRENT_SOURCE_DIR}/fast_double_parser-config.cmake.in"
"${CMAKE_CURRENT_BINARY_DIR}/fast_double_parser-config.cmake"
INSTALL_DESTINATION "share/fast_double_parser"
)
install(
FILES "${CMAKE_CURRENT_BINARY_DIR}/fast_double_parser-config.cmake"
DESTINATION "share/fast_double_parser"
)
option(BUILD_TESTING "Build unit tests" ON)
if(BUILD_TESTING)
add_executable(unit ${unit_src} ${bogus_src} ${rebogus_src})
if(FAST_DOUBLE_PARSER_SANITIZE)
target_compile_options(unit PUBLIC -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=all)
target_link_options(unit PUBLIC -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=all)
# Ubuntu bug for GCC 5.0+ (safe for all versions)
if (CMAKE_COMPILER_IS_GNUCC AND NOT APPLE)
target_link_libraries(unit PUBLIC -fuse-ld=gold)
endif()
endif()
target_link_libraries(unit PRIVATE fast_double_parser)
enable_testing()
add_test(unit unit)
endif()
target_link_libraries(unit PUBLIC fast_double_parser)
option(FAST_DOUBLE_BENCHMARKS "include benchmarks" OFF)
enable_testing()
add_test(unit unit)
option(FAST_DOUBLE_BENCHMARKS "Sanitize addresses" OFF)
function(initialize_submodule DIRECTORY)
if(NOT EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${DIRECTORY}/.git)
find_package(Git QUIET REQUIRED)
message(STATUS "${CMAKE_CURRENT_SOURCE_DIR}/${DIRECTORY}/.git does not exist. Initializing ${DIRECTORY} submodule ...")
execute_process(COMMAND ${GIT_EXECUTABLE} submodule update --init ${CMAKE_CURRENT_SOURCE_DIR}/${DIRECTORY}
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
RESULT_VARIABLE GIT_EXIT_CODE)
if(NOT GIT_EXIT_CODE EQUAL "0")
message(FATAL_ERROR "${GIT_EXECUTABLE} submodule update --init dependencies/${DIRECTORY} failed with exit code ${GIT_EXIT_CODE}, please checkout submodules")
endif()
endif()
endfunction(initialize_submodule)
if(FAST_DOUBLE_BENCHMARKS)
initialize_submodule(benchmarks/dependencies/abseil-cpp)
initialize_submodule(benchmarks/dependencies/double-conversion)
add_subdirectory(benchmarks/dependencies/abseil-cpp)
add_subdirectory(benchmarks/dependencies/double-conversion)
include(FetchContent)
include(ExternalProject)
set(ABSL_ENABLE_INSTALL ON)
set(ABSL_RUN_TEST OFF CACHE INTERNAL "")
set(ABSL_USE_GOOGLETEST_HEAD OFF CACHE INTERNAL "")
FetchContent_Declare(abseil
GIT_REPOSITORY https://github.com/abseil/abseil-cpp.git
GIT_TAG "20210324.2")
FetchContent_GetProperties(abseil)
if(NOT abseil_POPULATED)
set(BUILD_TESTING OFF)
FetchContent_Populate(abseil)
add_subdirectory(${abseil_SOURCE_DIR} ${abseil_BINARY_DIR})
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${abseil_SOURCE_DIR}/absl/copts)
include(${abseil_SOURCE_DIR}/absl/copts/AbseilConfigureCopts.cmake)
endif()
add_executable(benchmark ${benchmark_src})
target_link_libraries(benchmark PUBLIC double-conversion absl_strings)
target_include_directories(benchmark PUBLIC include)
endif(FAST_DOUBLE_BENCHMARKS)
FetchContent_Declare(doubleconversion
GIT_REPOSITORY https://github.com/google/double-conversion.git
GIT_TAG "v3.1.5")
FetchContent_GetProperties(doubleconversion)
FetchContent_MakeAvailable(doubleconversion)
add_executable(benchmark ${benchmark_src})
target_link_libraries(benchmark PUBLIC double-conversion absl::strings)
target_include_directories(benchmark PUBLIC include)
endif(FAST_DOUBLE_BENCHMARKS)

10
MODULE.bazel Normal file
View file

@ -0,0 +1,10 @@
"""fast_double_parser: 4x faster than strtod."""
module(
name = "fast_double_parser",
version = "0.8.0",
compatibility_level = 0,
)
bazel_dep(name = "abseil-cpp", version = "20240722.0", dev_dependency = True)
bazel_dep(name = "double-conversion", version = "3.3.0", dev_dependency = True)

View file

@ -1,17 +1,20 @@
# fast_double_parser: 4x faster than strtod
[![Build Status](https://cloud.drone.io/api/badges/lemire/fast_double_parser/status.svg)](https://cloud.drone.io/lemire/fast_double_parser) [![Build status](https://ci.appveyor.com/api/projects/status/y7215jgem4ggswnj/branch/master?svg=true)](https://ci.appveyor.com/project/lemire/fast-double-parser/branch/master)![VS16-CI](https://github.com/lemire/fast_double_parser/workflows/VS16-CI/badge.svg)![Ubuntu 18.04 CI (GCC 7)](https://github.com/lemire/fast_double_parser/workflows/Ubuntu%2018.04%20CI%20(GCC%207)/badge.svg)![VS16-Ninja-CI](https://github.com/lemire/fast_double_parser/workflows/VS16-Ninja-CI/badge.svg)![MSYS2-CI](https://github.com/lemire/fast_double_parser/workflows/MSYS2-CI/badge.svg)![VS16-CLANG-CI](https://github.com/lemire/fast_double_parser/workflows/VS16-CLANG-CI/badge.svg)![MinGW32-CI](https://github.com/lemire/fast_double_parser/workflows/MinGW32-CI/badge.svg)![MinGW64-CI](https://github.com/lemire/fast_double_parser/workflows/MinGW64-CI/badge.svg)![Ubuntu 20.04 CI (GCC 9)](https://github.com/lemire/fast_double_parser/workflows/Ubuntu%2020.04%20CI%20(GCC%209)/badge.svg)[![Build Status](https://api.cirrus-ci.com/github/lemire/fast_double_parser.svg)](https://cirrus-ci.com/github/lemire/fast_double_parser)
![MSYS2-CI](https://github.com/lemire/fast_double_parser/workflows/MSYS2-CI/badge.svg)[![Ubuntu 22.04](https://github.com/lemire/fast_double_parser/actions/workflows/ubuntu.yml/badge.svg)](https://github.com/lemire/fast_double_parser/actions/workflows/ubuntu.yml)
Unless you need support for [RFC 7159](https://tools.ietf.org/html/rfc7159) (JSON standard), we encourage users to adopt [fast_float](https://github.com/fastfloat/fast_float) library instead. It has more functionality.
Fast function to parse strings containing decimal numbers into double-precision (binary64) floating-point values. That is, given the string "1.0e10", it should return a 64-bit floating-point value equal to 10000000000. We do not sacrifice accuracy. The function will match exactly (down the smallest bit) the result of a standard function like `strtod`.
Fast function to parse ASCII strings containing decimal numbers into double-precision (binary64) floating-point values. That is, given the string "1.0e10", it should return a 64-bit floating-point value equal to 10000000000. We do not sacrifice accuracy. The function will match exactly (down the smallest bit) the result of a standard function like `strtod`.
We support all major compilers: Visual Studio, GNU GCC, LLVM Clang. We require C++11.
The core of this library was ported to Go by Nigel Tao and is now a standard float-parsing routine in Go (`strconv.ParseFloat`).
The core of this library was ported to Go by Nigel Tao and is now a standard float-parsing routine in Go (`strconv.ParseFloat`).
## Reference
- Daniel Lemire, [Number Parsing at a Gigabyte per Second](https://arxiv.org/abs/2101.11408), arXiv:2101.11408
- Daniel Lemire, [Number Parsing at a Gigabyte per Second](https://arxiv.org/abs/2101.11408), Software: Practice and Experience 51 (8), 2021.
## Usage
@ -27,13 +30,13 @@ The current API is simple enough:
double x;
char * string = ...
bool isok = fast_double_parser::parse_number(string, &x);
const char * endptr = fast_double_parser::parse_number(string, &x);
```
You must check the value returned (`isok`): if it is `nullptr`, then the function refused to parse the input.
You must check the value returned (`endptr`): if it is `nullptr`, then the function refused to parse the input.
Otherwise, we return a pointer (`const char *`) to the end of the parsed string. The provided
pointer (`string`) should point at the beginning of the number: if you must skip whitespace characters,
it is your responsability to do so.
it is your responsibility to do so.
We expect string numbers to follow [RFC 7159](https://tools.ietf.org/html/rfc7159) (JSON standard). In particular,
@ -44,10 +47,13 @@ It works much like the C standard function `strtod` expect that the parsing is l
1/2 even if you are under a French system. Locale independence is by design (it is not a limitation). Like the standard C functions, it expects that the string
representation of your number ends with a non-number character (e.g., a null character, a space, a colon, etc.). If you wish the specify the end point of the string, as is common in C++, please consider the [fast_float](https://github.com/lemire/fast_float) C++ library instead.
We assume that the rounding mode is set to nearest, the default setting (`std::fegetround() == FE_TONEAREST`). It is uncommon to have a different setting.
## What if I prefer another API?
The [fast_float](https://github.com/lemire/fast_float) offers an API resembling that of the C++17 `std::from_chars` functions. In particular, you can specify the beginning and the end of the string.
Furthermore [fast_float](https://github.com/lemire/fast_float) supports both 32-bit and 64-bit floating-point numbers. The [fast_float](https://github.com/lemire/fast_float) library is part of Apache Arrow.
Furthermore [fast_float](https://github.com/lemire/fast_float) supports both 32-bit and 64-bit floating-point numbers. The [fast_float](https://github.com/lemire/fast_float) library is part of Apache Arrow, GCC 12, Safari/WebKit and other important systems.
## Why should I expect this function to be faster?
@ -154,6 +160,7 @@ double-conv 243.90 MB/s
## Ports and users
- The algorithm is part of the [LLVM standard libraries](https://github.com/llvm/llvm-project/commit/87c016078ad72c46505461e4ff8bfa04819fe7ba).
- This library has been ported to Go and integrated in the Go standard library.
- It is part of the [Microsoft LightGBM machine-learning framework](https://github.com/microsoft/LightGBM).
- It is part of the database engine [noisepage](https://github.com/cmu-db/noisepage).
@ -162,6 +169,7 @@ double-conv 243.90 MB/s
- [There is a Rust port](https://github.com/ezrosent/frawk/tree/master/src/runtime/float_parse).
- [There is a Java port](https://github.com/wrandelshofer/FastDoubleParser).
- [There is a C# port](https://github.com/CarlVerret/csFastFloat).
- [Bazel Central Registry](https://registry.bazel.build/modules/fast_double_parser).
## Credit

View file

@ -20,7 +20,7 @@
#include <vector>
#include "double-conversion/ieee.h"
#include "double-conversion/string-to-double.h"
#include "double-conversion/double-conversion.h"
double findmax_fast_double_parser(const std::vector<std::string>& s) {
double answer = 0;

View file

@ -0,0 +1,3 @@
@PACKAGE_INIT@
include("${CMAKE_CURRENT_LIST_DIR}/fast_double_parser-targets.cmake")

View file

@ -190,28 +190,26 @@ really_inline value128 full_multiplication(uint64_t value1, uint64_t value2) {
return answer;
}
/* result might be undefined when input_num is zero */
inline int leading_zeroes(uint64_t input_num) {
#ifdef _MSC_VER
unsigned long leading_zero = 0;
// Search the mask data from most significant bit (MSB)
// to least significant bit (LSB) for a set bit (1).
#ifdef _WIN64
if (_BitScanReverse64(&leading_zero, input_num))
return (int)(63 - leading_zero);
else
return 64;
#else
if (_BitScanReverse(&leading_zero, (uint32_t)(input_num >> 32)))
return (int)(63 - (leading_zero + 32));
if (_BitScanReverse(&leading_zero, (uint32_t)input_num))
return (int)(63 - leading_zero);
#endif // _WIN64
#else
return __builtin_clzll(input_num);
#endif // _MSC_VER
}
// Precomputed powers of ten from 10^0 to 10^22. These
// can be represented exactly using the double type.
static const double power_of_ten[] = {
1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9, 1e10, 1e11,
1e12, 1e13, 1e14, 1e15, 1e16, 1e17, 1e18, 1e19, 1e20, 1e21, 1e22};
static inline bool is_integer(char c) {
return (c >= '0' && c <= '9');
// this gets compiled to (uint8_t)(c - '0') <= 9 on all decent compilers
@ -229,13 +227,28 @@ static inline bool is_integer(char c) {
* affect the binary significand.
*/
// The mantissas of powers of ten from -308 to 308, extended out to sixty four
// bits. The array contains the powers of ten approximated
// as a 64-bit mantissa. It goes from 10^FASTFLOAT_SMALLEST_POWER to
// 10^FASTFLOAT_LARGEST_POWER (inclusively).
// The mantissa is truncated, and
// never rounded up. Uses about 5KB.
static const uint64_t mantissa_64[] = {
// Attempts to compute i * 10^(power) exactly; and if "negative" is
// true, negate the result.
// This function will only work in some cases, when it does not work, success is
// set to false. This should work *most of the time* (like 99% of the time).
// We assume that power is in the [FASTFLOAT_SMALLEST_POWER,
// FASTFLOAT_LARGEST_POWER] interval: the caller is responsible for this check.
really_inline double compute_float_64(int64_t power, uint64_t i, bool negative,
bool *success) {
// Precomputed powers of ten from 10^0 to 10^22. These
// can be represented exactly using the double type.
static const double power_of_ten[] = {
1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9, 1e10, 1e11,
1e12, 1e13, 1e14, 1e15, 1e16, 1e17, 1e18, 1e19, 1e20, 1e21, 1e22};
// The mantissas of powers of ten from -308 to 308, extended out to sixty four
// bits. The array contains the powers of ten approximated
// as a 64-bit mantissa. It goes from 10^FASTFLOAT_SMALLEST_POWER to
// 10^FASTFLOAT_LARGEST_POWER (inclusively).
// The mantissa is truncated, and
// never rounded up. Uses about 5KB.
static const uint64_t mantissa_64[] = {
0xa5ced43b7e3e9188, 0xcf42894a5dce35ea,
0x818995ce7aa0e1b2, 0xa1ebfb4219491a1f,
0xca66fa129f9b60a6, 0xfd00b897478238d0,
@ -553,10 +566,10 @@ static const uint64_t mantissa_64[] = {
0xbaa718e68396cffd, 0xe950df20247c83fd,
0x91d28b7416cdd27e, 0xb6472e511c81471d,
0xe3d8f9e563a198e5, 0x8e679c2f5e44ff8f};
// A complement to mantissa_64
// complete to a 128-bit mantissa.
// Uses about 5KB but is rarely accessed.
const uint64_t mantissa_128[] = {
// A complement to mantissa_64
// complete to a 128-bit mantissa.
// Uses about 5KB but is rarely accessed.
static const uint64_t mantissa_128[] = {
0x419ea3bd35385e2d, 0x52064cac828675b9,
0x7343efebd1940993, 0x1014ebe6c5f90bf8,
0xd41a26e077774ef6, 0x8920b098955522b4,
@ -875,16 +888,6 @@ const uint64_t mantissa_128[] = {
0x4cdc331d57fa5441, 0xe0133fe4adf8e952,
0x58180fddd97723a6, 0x570f09eaa7ea7648,};
// Attempts to compute i * 10^(power) exactly; and if "negative" is
// true, negate the result.
// This function will only work in some cases, when it does not work, success is
// set to false. This should work *most of the time* (like 99% of the time).
// We assume that power is in the [FASTFLOAT_SMALLEST_POWER,
// FASTFLOAT_LARGEST_POWER] interval: the caller is responsible for this check.
really_inline double compute_float_64(int64_t power, uint64_t i, bool negative,
bool *success) {
// we start with a fast path
// It was described in
// Clinger WD. How to read floating point numbers accurately.
@ -1101,6 +1104,8 @@ static const char * parse_float_strtod(const char *ptr, double *outDouble) {
#elif defined(_WIN32)
static _locale_t c_locale = _create_locale(LC_ALL, "C");
*outDouble = _strtod_l(ptr, &endptr, c_locale);
#elif defined(__PASE__)
*outDouble = strtod(ptr, &endptr);
#else
static locale_t c_locale = newlocale(LC_ALL_MASK, "C", NULL);
*outDouble = strtod_l(ptr, &endptr, c_locale);

View file

@ -226,6 +226,16 @@ void issue32() {
std::cout << "zero maps to zero" << std::endl;
}
void negative_subsubnormal_to_negative_zero() {
std::string a = "-1e-999";
double x;
bool ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("could not parse -1e-999");
if(!std::signbit(x)) throw std::runtime_error("-1e-999 signbit is positive");
if(x != -0) throw std::runtime_error("-1e-999 is not -0");
std::cout << "-1e-999 parses to -0" << std::endl;
}
void issue50_fastpath() {
std::string a = "-0.0";
double x;
@ -235,6 +245,7 @@ void issue50_fastpath() {
std::cout << "-0.0 signbit is negative" << std::endl;
}
void issue50_off_fastpath() {
std::string a = "-0.0e-22";
double x;
@ -263,6 +274,27 @@ void issue23_2() {
std::cout << "can parse 5e0012" << std::endl;
}
void issue63() {
std::string a = "1-4-abc";
double x;
const char * ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("1-4-abc");
if(ok != a.c_str() + 1) throw std::runtime_error("does not point at the next number");
if(x != 1) throw std::runtime_error("cannot parse 1.");
std::cout << "1-4-abc" << std::endl;
}
void issue2093() {
std::string a = "0.95000000000000000000";
double x;
const char * ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("0.95000000000000000000");
if(x != 0.95) throw std::runtime_error("cannot parse 0.95000000000000000000.");
std::cout << "0.95000000000000000000" << std::endl;
}
inline void Assert(bool Assertion) {
if (!Assertion)
throw std::runtime_error("bug");
@ -307,11 +339,13 @@ int main() {
printf("Aborting further tests.");
return EXIT_SUCCESS;
}
issue2093();
Assert(basic_test_64bit("1090544144181609348835077142190",0x1.b8779f2474dfbp+99));
Assert(basic_test_64bit("4503599627370496.5", 4503599627370496.5));
Assert(basic_test_64bit("4503599627370497.5", 4503599627370497.5));
Assert(basic_test_64bit("0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000044501477170144022721148195934182639518696390927032912960468522194496444440421538910330590478162701758282983178260792422137401728773891892910553144148156412434867599762821265346585071045737627442980259622449029037796981144446145705102663115100318287949527959668236039986479250965780342141637013812613333119898765515451440315261253813266652951306000184917766328660755595837392240989947807556594098101021612198814605258742579179000071675999344145086087205681577915435923018910334964869420614052182892431445797605163650903606514140377217442262561590244668525767372446430075513332450079650686719491377688478005309963967709758965844137894433796621993967316936280457084866613206797017728916080020698679408551343728867675409720757232455434770912461317493580281734466552734375", 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000044501477170144022721148195934182639518696390927032912960468522194496444440421538910330590478162701758282983178260792422137401728773891892910553144148156412434867599762821265346585071045737627442980259622449029037796981144446145705102663115100318287949527959668236039986479250965780342141637013812613333119898765515451440315261253813266652951306000184917766328660755595837392240989947807556594098101021612198814605258742579179000071675999344145086087205681577915435923018910334964869420614052182892431445797605163650903606514140377217442262561590244668525767372446430075513332450079650686719491377688478005309963967709758965844137894433796621993967316936280457084866613206797017728916080020698679408551343728867675409720757232455434770912461317493580281734466552734375));
Assert(basic_test_64bit("0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022250738585072008890245868760858598876504231122409594654935248025624400092282356951787758888037591552642309780950434312085877387158357291821993020294379224223559819827501242041788969571311791082261043971979604000454897391938079198936081525613113376149842043271751033627391549782731594143828136275113838604094249464942286316695429105080201815926642134996606517803095075913058719846423906068637102005108723282784678843631944515866135041223479014792369585208321597621066375401613736583044193603714778355306682834535634005074073040135602968046375918583163124224521599262546494300836851861719422417646455137135420132217031370496583210154654068035397417906022589503023501937519773030945763173210852507299305089761582519159720757232455434770912461317493580281734466552734375", 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022250738585072008890245868760858598876504231122409594654935248025624400092282356951787758888037591552642309780950434312085877387158357291821993020294379224223559819827501242041788969571311791082261043971979604000454897391938079198936081525613113376149842043271751033627391549782731594143828136275113838604094249464942286316695429105080201815926642134996606517803095075913058719846423906068637102005108723282784678843631944515866135041223479014792369585208321597621066375401613736583044193603714778355306682834535634005074073040135602968046375918583163124224521599262546494300836851861719422417646455137135420132217031370496583210154654068035397417906022589503023501937519773030945763173210852507299305089761582519159720757232455434770912461317493580281734466552734375));
issue63();
issue40();
issue23();
issue32();
@ -360,6 +394,7 @@ int main() {
throw std::runtime_error("fast_double_parser disagrees");
}
}
negative_subsubnormal_to_negative_zero();
std::cout << std::endl;
std::cout << "All ok" << std::endl;
printf("Good!\n");