Compare commits

...

93 commits

Author SHA1 Message Date
Daniel Lemire
bc93aee338
Merge pull request #80 from BYVoid/bench
Add benchmark binary to Bazel BUILD.
2024-10-31 11:14:09 -04:00
Carbo Kuo
f5096c7a94 Add benchmark binary to Bazel BUILD.
To build it, run:
bazel build --compilation_mode=opt //:benchmark

To run the benchmark, run:
bazel-bin/benchmark benchmarks/data/canada.txt

Result from my computer:

read 111126 lines

=== trial 1 ===
fast_double_parser  1315.70 MB/s
strtod         722.10 MB/s
abslfromch     810.26 MB/s
absl           799.89 MB/s
double-conv    396.05 MB/s

=== trial 2 ===
fast_double_parser  1416.58 MB/s
strtod         751.43 MB/s
abslfromch     841.83 MB/s
absl           838.71 MB/s
double-conv    415.33 MB/s
2024-10-31 10:39:29 -04:00
Daniel Lemire
22ac46158c
Merge pull request #79 from BYVoid/bazel-build
Add a simple Bazel module and build file.
2024-10-31 09:48:55 -04:00
Carbo Kuo
3aaddfe336 Add a simple Bazel module and build file. 2024-10-31 08:44:43 -04:00
Daniel Lemire
252029ddac
Update README.md 2024-05-07 10:01:17 -04:00
Daniel Lemire
d971fa76ee update 2024-05-07 09:59:45 -04:00
Daniel Lemire
305e6c8170 better cmake 2024-03-18 10:17:36 -04:00
Daniel Lemire
d61b589f65 tuning. 2024-03-18 10:03:14 -04:00
Daniel Lemire
b92d89b2c4 updating 2024-03-18 10:01:21 -04:00
Daniel Lemire
ca05d13e26
Merge pull request #77 from Osyotr/cmake-exports
Export cmake config and add option to build unit tests
2024-03-18 09:03:29 -04:00
Osyotr
2a3319d698 Export cmake config and add option to build unit tests 2024-03-17 22:45:44 +03:00
Osyotr
468ee515f4 Add visual studio files to gitignore 2024-03-17 22:18:48 +03:00
Daniel Lemire
172f42bccf
Merge pull request #76 from barracuda156/apple
CMakeLists: do not use Linux linker flag on macOS
2024-03-12 13:15:34 -04:00
Sergey Fedorov
da4ba02ea7
CMakeLists: do not use Linux linker flag on macOS 2024-03-10 01:17:32 +07:00
Daniel Lemire
4f5e530b02
Update README.md 2023-12-26 16:06:21 -05:00
Daniel Lemire
15f93a61bb
Update README.md 2023-12-26 16:05:15 -05:00
Daniel Lemire
3489ead54c
Merge pull request #75 from BebeSparkelSparkel/master
Deprecation notice at top
2023-12-26 16:04:07 -05:00
William Rusnack
b94e499b22
Deprecation notice at top 2023-12-26 11:16:07 -05:00
Daniel Lemire
d2e8f223e7
Merge pull request #74 from DavidKorczynski/clusterfuzzlite-integration
Add ClusterFuzzLite integration
2023-12-21 09:48:50 -05:00
David Korczynski
06bc1ea3ed Add ClusterFuzzLite integration
Signed-off-by: David Korczynski <david@adalogics.com>
2023-12-21 03:48:47 -08:00
Daniel Lemire
38c6d6da54 removing old ci 2023-12-06 21:15:13 -05:00
Daniel Lemire
56a813c54f added test. 2023-12-06 21:14:43 -05:00
Daniel Lemire
cee3272a5c
Update README.md 2023-12-02 18:54:05 -05:00
Daniel Lemire
07d9189a8f
Merge pull request #72 from striezel-stash/update-actions-setup-cmake
Update jwlawson/actions-setup-cmake in GitHub Actions to v1.13
2023-01-10 10:24:35 -05:00
Dirk Stolle
8a55baef01 Update jwlawson/actions-setup-cmake in GitHub Actions to v1.13 2023-01-09 22:56:33 +01:00
Daniel Lemire
d039d6fe5e
Merge pull request #71 from nxtn-staged/patch-1
Use _BitScanReverse on Win32
2023-01-03 09:50:49 -05:00
nxtn-staged
5a8c7606df
Use _BitScanReverse on Win32 2022-12-27 11:23:19 +08:00
Daniel Lemire
efec03532e
Merge pull request #67 from lgtm-migrator/codeql
Add CodeQL workflow for GitHub code scanning
2022-12-08 10:20:27 -05:00
Daniel Lemire
027fb3abd3
Merge pull request #69 from striezel-stash/action-update
Update actions/checkout in GitHub Actions to v3
2022-12-08 10:20:15 -05:00
Daniel Lemire
749ba90831
Merge pull request #70 from zheddie/master
Supporting IBM i/PASE env.
2022-12-08 10:20:02 -05:00
GavinZhang
0e2a13868f Refine code according to the comments from reviewer. 2022-12-08 17:10:53 +08:00
GavinZhang
2df5ff039b Supporting IBM i/PASE env. 2022-12-07 17:09:20 +08:00
Daniel Lemire
7aebf2f747
Merge pull request #68 from striezel-stash/readme-typo
Fix a typo in README.md
2022-11-15 18:25:41 -05:00
Dirk Stolle
1427971755 Update actions/checkout in GitHub Actions to v3 2022-11-16 00:19:07 +01:00
Dirk Stolle
4a73a889f0 Fix a typo in README.md 2022-11-16 00:13:34 +01:00
LGTM Migrator
e8c1bf481e
Add CodeQL workflow for GitHub code scanning 2022-11-10 21:03:54 +00:00
Daniel Lemire
66657d1a48
Update README.md 2022-11-03 19:15:38 -04:00
Daniel Lemire
357eec4e61 Adding test. 2022-10-02 10:59:26 -04:00
Daniel Lemire
267b118aec
Update README.md 2022-07-24 12:44:28 -04:00
Daniel Lemire
b823bbc8a0
Merge pull request #64 from lemire/dlemire/issue63
Adding test for issue 63.
2022-07-09 13:47:17 -04:00
Daniel Lemire
c3b92a47c7 Adding test for issue 63. 2022-07-09 13:39:27 -04:00
Daniel Lemire
50a2ccb553
Update README.md 2022-05-19 12:32:19 -04:00
Daniel Lemire
4ca9eb3221
Merge pull request #62 from lemire/dlemire/upgradingtovs17
Upgrading tests to latest visual studio
2022-05-19 12:31:07 -04:00
Daniel Lemire
3b296fc600 Fixing legacy Win32 support 2022-05-19 12:25:13 -04:00
Daniel Lemire
182c61b1ce Removing mingw 2022-05-19 09:54:31 -04:00
Daniel Lemire
8ed6508c0d Upgrading to latest visual studio 2022-05-19 09:52:42 -04:00
Daniel Lemire
36f2c5ef59
Merge pull request #61 from ruggiero/master
Fix x32 variable
2022-05-19 09:46:04 -04:00
Rafael Ruggiero
b3c7fce960
Fix x32 variable 2022-05-19 02:30:15 -03:00
Daniel Lemire
ffe3619894
Update README.md 2022-03-07 09:47:50 -05:00
Daniel Lemire
111ad80417
Update README.md 2021-10-23 18:04:54 -04:00
Daniel Lemire
30302eac8c
Update README.md 2021-07-02 14:18:26 -04:00
Daniel Lemire
715cf49f1a
Merge pull request #56 from lemire/issue49
Patch for legacy systems
2021-05-18 14:40:50 -04:00
Daniel Lemire
4d1ada99c2 Removing para 2021-05-18 14:33:25 -04:00
Daniel Lemire
70f4b8fac1 Patch. 2021-05-18 14:31:23 -04:00
Daniel Lemire
2cc127528c
Update CMakeLists.txt 2021-05-18 13:54:18 -04:00
Daniel Lemire
fa50da4ffb
Update README.md 2021-04-23 09:23:01 -04:00
Daniel Lemire
e4f6319bfa
Merge pull request #54 from wojdyr/b1
move arrays from file scope to compute_float_64()
2021-04-01 13:07:37 -04:00
Marcin Wojdyr
b40d08a245 move arrays from file scope to compute_float_64()
It avoids duplication of the arrays in each compilation unit,
saving about (N-1)*10Kb in the binary size.
2021-04-01 18:38:59 +02:00
Daniel Lemire
c0b367096a
Merge pull request #52 from wojdyr/patch-1
README.md: fix parse_number return type
2021-03-31 17:48:05 -04:00
Marcin Wojdyr
cf0f3daeaa
README.md: fix parse_number return type 2021-03-31 23:29:24 +02:00
Daniel Lemire
47d76a06f5
Merge pull request #51 from swankjesse/jwilson.0321.negative_zero
Fix an unlikely corner case for negative zero
2021-03-21 12:30:09 -04:00
Jesse Wilson
f356e099ba Fix an unlikely corner case for negative zero
Closes: https://github.com/lemire/fast_double_parser/issues/50
2021-03-21 09:35:05 -04:00
Daniel Lemire
6d4b4a5df8
Update README.md 2021-03-20 15:52:13 -04:00
Daniel Lemire
933a592fab
Update README.md 2021-01-29 12:28:31 -05:00
Daniel Lemire
d52e3059bd
Update table_generation.py 2021-01-13 09:17:53 -05:00
Daniel Lemire
6a47c2a5a1
Merge pull request #46 from lemire/dlemire/more_ci
Adding clang/msys
2020-12-22 15:22:19 -05:00
Daniel Lemire
687e40f586 Merge branch 'master' into dlemire/more_ci 2020-12-22 15:04:37 -05:00
Daniel Lemire
becce11e8d
Merge pull request #47 from lemire/dlemire/stringstream_patch
Minor patch to backup stringstream approach.
2020-12-22 15:03:38 -05:00
Daniel Lemire
4bdc4dbd7c Minor patch to backup stringstream approach. 2020-12-22 14:39:03 -05:00
Daniel Lemire
d4c113a67d Specify the compiler. 2020-12-22 10:01:36 -05:00
Daniel Lemire
5fe4363d26 Adding clang/msys. 2020-12-22 09:56:20 -05:00
Daniel Lemire
c607ead369
Update README.md 2020-12-08 09:26:00 -05:00
Daniel Lemire
589ca58384
Merge pull request #45 from lemire/dlemire/updatingaction
Updating actions
2020-11-24 14:24:34 -05:00
Daniel Lemire
5a4c73f089 Updating action. 2020-11-24 10:35:03 -05:00
Daniel Lemire
e078c17d2d
Update README.md 2020-11-24 09:18:31 -05:00
Daniel Lemire
5eaa0ad0a4
Update README.md 2020-11-13 15:14:41 -05:00
Daniel Lemire
9e6fcfc38b Table generation script. 2020-11-10 11:58:26 -05:00
Daniel Lemire
ace60646c0
Merge pull request #44 from lemire/dlemire/patch_solaris
Extending the fallback.
2020-11-10 10:43:52 -05:00
Daniel Lemire
eb16768ad9 Tweaks. 2020-11-10 10:15:25 -05:00
Daniel Lemire
5c8d46cc09 Let us try solaris CI. 2020-11-10 09:56:40 -05:00
Daniel Lemire
6f5fee819e Extending the fallback. 2020-11-10 09:49:55 -05:00
Daniel Lemire
94faf8de8a
Merge pull request #43 from lemire/dlemire/fix_issue_42
fix issue 42
2020-11-08 11:34:45 -05:00
Daniel Lemire
159342d8af More issues. 2020-11-08 11:13:05 -05:00
Daniel Lemire
ce01fe601c Merge branch 'master' into dlemire/fix_issue_42 2020-11-08 11:08:23 -05:00
Daniel Lemire
278882dd58 This fixes issue 42. 2020-11-08 11:05:52 -05:00
Daniel Lemire
7496db7531 Tweaking documentation. 2020-11-07 16:31:52 -05:00
Daniel Lemire
8e3afac2d8
Merge pull request #41 from lemire/dlemire/moving_a_variable
Added remarks.
2020-11-03 10:13:37 -05:00
Daniel Lemire
2905c0e79d
Merge branch 'master' into dlemire/moving_a_variable 2020-11-03 10:13:06 -05:00
Daniel Lemire
42c5413e8e Added remarks. 2020-11-03 10:09:00 -05:00
Daniel Lemire
b97a589859
Merge pull request #39 from lemire/dlemire/moving_a_variable
moving a variable
2020-11-02 20:04:37 -05:00
Daniel Lemire
8214b37c86 This moves a variable to clean things up. 2020-11-02 14:44:24 -05:00
Daniel Lemire
b4ea262021 Added secondary license. 2020-10-26 16:49:38 -04:00
Daniel Lemire
1f6873e931 Adding a new test. 2020-10-23 11:02:29 -04:00
33 changed files with 616 additions and 302 deletions

View file

@ -22,9 +22,9 @@ build_script:
- mkdir build
- cd build
- cmake --version
- cmake %CMAKE_ARGS% --parallel ..
- cmake %CMAKE_ARGS% ..
- cmake ..
- cmake --build . --config %Configuration% --verbose --parallel
- cmake --build . --config %Configuration% --verbose
for:
-
@ -33,7 +33,7 @@ for:
- job_name: VS2019ARM
test_script:
- ctest --output-on-failure -C %Configuration% --verbose %CTEST_ARGS% --parallel
- ctest --output-on-failure -C %Configuration% --verbose %CTEST_ARGS%
clone_folder: c:\projects\fast_double_parser

View file

@ -0,0 +1,6 @@
FROM gcr.io/oss-fuzz-base/base-builder
RUN apt-get update && apt-get install -y make autoconf automake libtool
COPY . $SRC/fast_double_parser
COPY .clusterfuzzlite/build.sh $SRC/build.sh
WORKDIR $SRC/fast_double_parser

View file

@ -0,0 +1,3 @@
# ClusterFuzzLite set up
This folder contains a fuzzing set for [ClusterFuzzLite](https://google.github.io/clusterfuzzlite).

View file

@ -0,0 +1,7 @@
#!/bin/bash -eu
# Copy all fuzzer executable to $OUT/
$CXX $CFLAGS $LIB_FUZZING_ENGINE \
$SRC/fast_double_parser/.clusterfuzzlite/parse_number_fuzzer.cpp \
-o $OUT/parse_number_fuzzer \
-I$SRC/fast_double_parser/include

View file

@ -0,0 +1,11 @@
#include <fast_double_parser.h>
#include <string>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
std::string fuzz_input(reinterpret_cast<const char *>(data), size);
double x;
fast_double_parser::parse_number(fuzz_input.c_str(), &x);
return 0;
}

View file

@ -0,0 +1 @@
language: c++

30
.github/workflows/cflite_pr.yml vendored Normal file
View file

@ -0,0 +1,30 @@
name: ClusterFuzzLite PR fuzzing
on:
workflow_dispatch:
pull_request:
branches: [ main ]
permissions: read-all
jobs:
PR:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sanitizer: [address]
steps:
- name: Build Fuzzers (${{ matrix.sanitizer }})
id: build
uses: google/clusterfuzzlite/actions/build_fuzzers@v1
with:
sanitizer: ${{ matrix.sanitizer }}
language: c++
bad-build-check: false
- name: Run Fuzzers (${{ matrix.sanitizer }})
id: run
uses: google/clusterfuzzlite/actions/run_fuzzers@v1
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
fuzz-seconds: 100
mode: 'code-change'
report-unreproducible-crashes: false
sanitizer: ${{ matrix.sanitizer }}

41
.github/workflows/codeql.yml vendored Normal file
View file

@ -0,0 +1,41 @@
name: "CodeQL"
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
schedule:
- cron: "1 8 * * 3"
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ cpp ]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
queries: +security-and-quality
- name: Autobuild
uses: github/codeql-action/autobuild@v2
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{ matrix.language }}"

View file

@ -1,49 +0,0 @@
name: MinGW32-CI
on: [push, pull_request]
# Important: scoop will either install 32-bit GCC or 64-bit GCC, not both.
# It is important to build static libraries because cmake is not smart enough under Windows/mingw to take care of the path. So
# with a dynamic library, you could get failures due to the fact that the EXE can't find its DLL.
jobs:
ci:
name: windows-gcc
runs-on: windows-2016
env:
CMAKE_GENERATOR: Ninja
CC: gcc
CXX: g++
steps: # To reproduce what is below, start a powershell with administrative rights, using scoop *is* a good idea
- uses: actions/checkout@v2
- uses: actions/cache@v2 # we cache the scoop setup with 32-bit GCC
id: cache
with:
path: |
C:\ProgramData\scoop
key: scoop32 # static key: should be good forever
- name: Setup Windows # This should almost never run if the cache works.
if: steps.cache.outputs.cache-hit != 'true'
shell: powershell
run: |
Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')
scoop install sudo --global
sudo scoop install git --global
sudo scoop install ninja --global
sudo scoop install cmake --global
sudo scoop install gcc --arch 32bit --global
$env:path
Write-Host 'Everything has been installed, you are good!'
- name: Build and Test 32-bit x86
shell: powershell
run: |
$ENV:PATH="C:\ProgramData\scoop\shims;C:\ProgramData\scoop\apps\gcc\current\bin;C:\ProgramData\scoop\apps\ninja\current;$ENV:PATH"
mkdir build32
cd build32
cmake ..
cmake --build . --verbose
ctest -j4 --output-on-failure

View file

@ -1,49 +0,0 @@
name: MinGW64-CI
on: [push, pull_request]
# Important: scoop will either install 32-bit GCC or 64-bit GCC, not both.
# It is important to build static libraries because cmake is not smart enough under Windows/mingw to take care of the path. So
# with a dynamic library, you could get failures due to the fact that the EXE can't find its DLL.
jobs:
ci:
name: windows-gcc
runs-on: windows-2016
env:
CMAKE_GENERATOR: Ninja
CC: gcc
CXX: g++
steps: # To reproduce what is below, start a powershell with administrative rights, using scoop *is* a good idea
- uses: actions/checkout@v2
- uses: actions/cache@v2 # we cache the scoop setup with 64-bit GCC
id: cache
with:
path: |
C:\ProgramData\scoop
key: scoop64 # static key: should be good forever
- name: Setup Windows # This should almost never run if the cache works.
if: steps.cache.outputs.cache-hit != 'true'
shell: powershell
run: |
Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')
scoop install sudo --global
sudo scoop install git --global
sudo scoop install ninja --global
sudo scoop install cmake --global
sudo scoop install gcc --arch 64bit --global
$env:path
Write-Host 'Everything has been installed, you are good!'
- name: Build and Test 64-bit x64
shell: powershell
run: |
$ENV:PATH="C:\ProgramData\scoop\shims;C:\ProgramData\scoop\apps\gcc\current\bin;C:\ProgramData\scoop\apps\ninja\current;$ENV:PATH"
mkdir build64
cd build64
cmake ..
cmake --build . --verbose
ctest -j4 --output-on-failure

44
.github/workflows/msys2-clang.yml vendored Normal file
View file

@ -0,0 +1,44 @@
name: MSYS2-CLANG-CI
on: [push, pull_request]
jobs:
windows-mingw:
name: ${{ matrix.msystem }}
runs-on: windows-latest
defaults:
run:
shell: msys2 {0}
strategy:
fail-fast: false
matrix:
include:
- msystem: "MINGW64"
install: mingw-w64-x86_64-cmake mingw-w64-x86_64-ninja mingw-w64-x86_64-clang
type: Release
- msystem: "MINGW32"
install: mingw-w64-i686-cmake mingw-w64-i686-ninja mingw-w64-i686-clang
type: Release
- msystem: "MINGW64"
install: mingw-w64-x86_64-cmake mingw-w64-x86_64-ninja mingw-w64-x86_64-clang
type: Debug
- msystem: "MINGW32"
install: mingw-w64-i686-cmake mingw-w64-i686-ninja mingw-w64-i686-clang
type: Debug
env:
CMAKE_GENERATOR: Ninja
steps:
- uses: actions/checkout@v3
- uses: msys2/setup-msys2@v2
with:
update: true
msystem: ${{ matrix.msystem }}
install: ${{ matrix.install }}
- name: Build and Test
run: |
mkdir build
cd build
cmake -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=${{ matrix.type }} ..
cmake --build . --verbose
ctest -j4 --output-on-failure

View file

@ -29,7 +29,7 @@ jobs:
CMAKE_GENERATOR: Ninja
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: msys2/setup-msys2@v2
with:
update: true

21
.github/workflows/rhub.yml vendored Normal file
View file

@ -0,0 +1,21 @@
name: rhub
'on':
- push
- pull_request
jobs:
ubuntu-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: start docker
run: |
docker run -w /src -dit --name rhub -v $PWD:/src rhub/rocker-gcc-san
echo 'docker exec rhub "$@";' > ./rhub.sh
chmod +x ./rhub.sh
- name: build
run: |
./rhub.sh c++ tests/unit.cpp -I include
- name: test
run: |
./rhub.sh ./a.out

View file

@ -1,16 +1,15 @@
name: Ubuntu 20.04 CI (GCC 9)
name: Ubuntu 22.04
on: [push, pull_request]
jobs:
ubuntu-build:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
strategy:
matrix:
cxx: [g++-12, clang++-14]
steps:
- uses: actions/checkout@v2
- name: Setup cmake
uses: jwlawson/actions-setup-cmake@v1.0
with:
cmake-version: '3.9.x'
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
- name: Use cmake
run: |
mkdir build &&

View file

@ -1,20 +0,0 @@
name: Ubuntu 18.04 CI (GCC 7)
on: [push, pull_request]
jobs:
ubuntu-build:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
- name: Setup cmake
uses: jwlawson/actions-setup-cmake@v1.0
with:
cmake-version: '3.9.x'
- name: Use cmake
run: |
mkdir build &&
cd build &&
cmake -DFAST_DOUBLE_BENCHMARKS=ON .. &&
cmake --build . &&
ctest -j --output-on-failure

View file

@ -1,24 +0,0 @@
name: VS16-CI
on: [push, pull_request]
jobs:
ci:
name: windows-vs16
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: 'Run CMake with VS16'
uses: lukka/run-cmake@v2
with:
cmakeListsOrSettingsJson: CMakeListsTxtAdvanced
cmakeListsTxtPath: '${{ github.workspace }}/CMakeLists.txt'
buildDirectory: "${{ github.workspace }}/../../_temp/windows"
cmakeBuildType: Release
buildWithCMake: true
cmakeGenerator: VS16Win64
buildWithCMakeArgs: --config Release
- name: 'Run CTest'
run: ctest -C Release --output-on-failure
working-directory: "${{ github.workspace }}/../../_temp/windows"

View file

@ -1,25 +0,0 @@
name: VS16-CLANG-CI
on: [push, pull_request]
jobs:
ci:
name: windows-vs16
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: 'Run CMake with VS16'
uses: lukka/run-cmake@v2
with:
cmakeListsOrSettingsJson: CMakeListsTxtAdvanced
cmakeListsTxtPath: '${{ github.workspace }}/CMakeLists.txt'
buildDirectory: "${{ github.workspace }}/../../_temp/windows"
cmakeBuildType: Release
buildWithCMake: true
cmakeGenerator: VS16Win64
cmakeAppendedArgs: -T ClangCL
buildWithCMakeArgs: --config Release
- name: 'Run CTest'
run: ctest -C Release --output-on-failure
working-directory: "${{ github.workspace }}/../../_temp/windows"

View file

@ -1,25 +0,0 @@
name: VS16-Ninja-CI
on: [push, pull_request]
jobs:
ci:
name: windows-vs16
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: 'Run CMake with VS16'
uses: lukka/run-cmake@v2
with:
cmakeListsOrSettingsJson: CMakeListsTxtAdvanced
cmakeListsTxtPath: '${{ github.workspace }}/CMakeLists.txt'
buildDirectory: "${{ github.workspace }}/../../_temp/windows"
cmakeBuildType: Release
buildWithCMake: true
cmakeGenerator: VS16Win64
cmakeAppendedArgs: -G Ninja
buildWithCMakeArgs: --config Release
- name: 'Run CTest'
run: ctest -C Release --output-on-failure
working-directory: "${{ github.workspace }}/../../_temp/windows"

37
.github/workflows/vs17-ci.yml vendored Normal file
View file

@ -0,0 +1,37 @@
name: VS17-CI
on: [push, pull_request]
jobs:
ci:
if: >-
! contains(toJSON(github.event.commits.*.message), '[skip ci]') &&
! contains(toJSON(github.event.commits.*.message), '[skip github]')
name: windows-vs17
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
include:
- {gen: Visual Studio 17 2022, arch: Win32}
- {gen: Visual Studio 17 2022, arch: Win32}
- {gen: Visual Studio 17 2022, arch: x64}
- {gen: Visual Studio 17 2022, arch: x64}
steps:
- name: checkout
uses: actions/checkout@v3
- name: Configure
run: |
cmake -G "${{matrix.gen}}" -A ${{matrix.arch}} -B build
- name: Build Debug
run: cmake --build build --config Debug --verbose
- name: Build Release
run: cmake --build build --config Release --verbose
- name: Run Release tests
run: |
cd build
ctest -C Release --output-on-failure
- name: Run Debug tests
run: |
cd build
ctest -C Debug --output-on-failure

37
.github/workflows/vs17-clang-ci.yml vendored Normal file
View file

@ -0,0 +1,37 @@
name: VS17-CLANG-CI
on: [push, pull_request]
jobs:
ci:
if: >-
! contains(toJSON(github.event.commits.*.message), '[skip ci]') &&
! contains(toJSON(github.event.commits.*.message), '[skip github]')
name: windows-vs17
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
include:
- {gen: Visual Studio 17 2022, arch: Win32}
- {gen: Visual Studio 17 2022, arch: Win32}
- {gen: Visual Studio 17 2022, arch: x64}
- {gen: Visual Studio 17 2022, arch: x64}
steps:
- name: checkout
uses: actions/checkout@v3
- name: Configure
run: |
cmake -G "${{matrix.gen}}" -A ${{matrix.arch}} -T ClangCL -B build
- name: Build Debug
run: cmake --build build --config Debug --verbose
- name: Build Release
run: cmake --build build --config Release --verbose
- name: Run Release tests
run: |
cd build
ctest -C Release --output-on-failure
- name: Run Debug tests
run: |
cd build
ctest -C Debug --output-on-failure

4
.gitignore vendored
View file

@ -44,3 +44,7 @@
/fast_double_parser.cxxflags
/fast_double_parser.files
/fast_double_parser.includes
# Visual Studio
/.vs
/out

22
BUILD.bazel Normal file
View file

@ -0,0 +1,22 @@
cc_library(
name = "fast_double_parser",
hdrs = ["include/fast_double_parser.h"],
strip_include_prefix = "include",
visibility = ["//visibility:public"],
)
cc_test(
name = "unit",
srcs = ["tests/unit.cpp"],
deps = [":fast_double_parser"],
)
cc_binary(
name = "benchmark",
srcs = ["benchmarks/benchmark.cpp"],
deps = [
":fast_double_parser",
"@abseil-cpp//absl/strings",
"@double-conversion",
],
)

View file

@ -1,6 +1,7 @@
cmake_minimum_required(VERSION 3.11)
cmake_policy(SET CMP0048 NEW)
project(fast_double_parser LANGUAGES CXX VERSION 0.0.0.0)
cmake_minimum_required(VERSION 3.9)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
if (NOT CMAKE_BUILD_TYPE)
@ -12,50 +13,89 @@ option(FAST_DOUBLE_PARSER_SANITIZE "Sanitize addresses" OFF)
set(headers include/fast_double_parser.h)
set(unit_src tests/unit.cpp)
set(bogus_src tests/bogus.cpp)
set(rebogus_src tests/bogus.cpp)
set(benchmark_src benchmarks/benchmark.cpp)
add_executable(unit ${unit_src})
if(FAST_DOUBLE_PARSER_SANITIZE)
target_compile_options(unit PUBLIC -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=all)
target_link_options(unit PUBLIC -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=all)
# Ubuntu bug for GCC 5.0+ (safe for all versions)
if (CMAKE_COMPILER_IS_GNUCC)
target_link_libraries(unit PUBLIC -fuse-ld=gold)
endif()
add_library(fast_double_parser INTERFACE)
target_include_directories(fast_double_parser
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}/include>
$<INSTALL_INTERFACE:include>
)
include(GNUInstallDirs)
install(FILES ${headers} DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}")
install(TARGETS fast_double_parser EXPORT fast_double_parser-targets)
install(
EXPORT fast_double_parser-targets
DESTINATION "share/fast_double_parser"
NAMESPACE fast_double_parser::
)
include(CMakePackageConfigHelpers)
configure_package_config_file(
"${CMAKE_CURRENT_SOURCE_DIR}/fast_double_parser-config.cmake.in"
"${CMAKE_CURRENT_BINARY_DIR}/fast_double_parser-config.cmake"
INSTALL_DESTINATION "share/fast_double_parser"
)
install(
FILES "${CMAKE_CURRENT_BINARY_DIR}/fast_double_parser-config.cmake"
DESTINATION "share/fast_double_parser"
)
option(BUILD_TESTING "Build unit tests" ON)
if(BUILD_TESTING)
add_executable(unit ${unit_src} ${bogus_src} ${rebogus_src})
if(FAST_DOUBLE_PARSER_SANITIZE)
target_compile_options(unit PUBLIC -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=all)
target_link_options(unit PUBLIC -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=all)
# Ubuntu bug for GCC 5.0+ (safe for all versions)
if (CMAKE_COMPILER_IS_GNUCC AND NOT APPLE)
target_link_libraries(unit PUBLIC -fuse-ld=gold)
endif()
endif()
target_link_libraries(unit PRIVATE fast_double_parser)
enable_testing()
add_test(unit unit)
endif()
target_include_directories(unit PUBLIC include)
enable_testing()
add_test(unit unit)
option(FAST_DOUBLE_BENCHMARKS "Sanitize addresses" OFF)
option(FAST_DOUBLE_BENCHMARKS "include benchmarks" OFF)
function(initialize_submodule DIRECTORY)
if(NOT EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${DIRECTORY}/.git)
find_package(Git QUIET REQUIRED)
message(STATUS "${CMAKE_CURRENT_SOURCE_DIR}/${DIRECTORY}/.git does not exist. Initializing ${DIRECTORY} submodule ...")
execute_process(COMMAND ${GIT_EXECUTABLE} submodule update --init ${CMAKE_CURRENT_SOURCE_DIR}/${DIRECTORY}
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
RESULT_VARIABLE GIT_EXIT_CODE)
if(NOT GIT_EXIT_CODE EQUAL "0")
message(FATAL_ERROR "${GIT_EXECUTABLE} submodule update --init dependencies/${DIRECTORY} failed with exit code ${GIT_EXIT_CODE}, please checkout submodules")
endif()
endif()
endfunction(initialize_submodule)
if(FAST_DOUBLE_BENCHMARKS)
initialize_submodule(benchmarks/dependencies/abseil-cpp)
initialize_submodule(benchmarks/dependencies/double-conversion)
add_subdirectory(benchmarks/dependencies/abseil-cpp)
add_subdirectory(benchmarks/dependencies/double-conversion)
include(FetchContent)
include(ExternalProject)
set(ABSL_ENABLE_INSTALL ON)
set(ABSL_RUN_TEST OFF CACHE INTERNAL "")
set(ABSL_USE_GOOGLETEST_HEAD OFF CACHE INTERNAL "")
FetchContent_Declare(abseil
GIT_REPOSITORY https://github.com/abseil/abseil-cpp.git
GIT_TAG "20210324.2")
FetchContent_GetProperties(abseil)
if(NOT abseil_POPULATED)
set(BUILD_TESTING OFF)
FetchContent_Populate(abseil)
add_subdirectory(${abseil_SOURCE_DIR} ${abseil_BINARY_DIR})
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${abseil_SOURCE_DIR}/absl/copts)
include(${abseil_SOURCE_DIR}/absl/copts/AbseilConfigureCopts.cmake)
endif()
add_executable(benchmark ${benchmark_src})
target_link_libraries(benchmark PUBLIC double-conversion absl_strings)
target_include_directories(benchmark PUBLIC include)
endif(FAST_DOUBLE_BENCHMARKS)
FetchContent_Declare(doubleconversion
GIT_REPOSITORY https://github.com/google/double-conversion.git
GIT_TAG "v3.1.5")
FetchContent_GetProperties(doubleconversion)
FetchContent_MakeAvailable(doubleconversion)
add_executable(benchmark ${benchmark_src})
target_link_libraries(benchmark PUBLIC double-conversion absl::strings)
target_include_directories(benchmark PUBLIC include)
endif(FAST_DOUBLE_BENCHMARKS)

25
LICENSE.BSL Normal file
View file

@ -0,0 +1,25 @@
Copyright (c) Daniel Lemire
Boost Software License - Version 1.0 - August 17th, 2003
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:
The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.

10
MODULE.bazel Normal file
View file

@ -0,0 +1,10 @@
"""fast_double_parser: 4x faster than strtod."""
module(
name = "fast_double_parser",
version = "0.8.0",
compatibility_level = 0,
)
bazel_dep(name = "abseil-cpp", version = "20240722.0", dev_dependency = True)
bazel_dep(name = "double-conversion", version = "3.3.0", dev_dependency = True)

View file

@ -1,12 +1,20 @@
# fast_double_parser
[![Build Status](https://cloud.drone.io/api/badges/lemire/fast_double_parser/status.svg)](https://cloud.drone.io/lemire/fast_double_parser) [![Build status](https://ci.appveyor.com/api/projects/status/y7215jgem4ggswnj/branch/master?svg=true)](https://ci.appveyor.com/project/lemire/fast-double-parser/branch/master)![VS16-CI](https://github.com/lemire/fast_double_parser/workflows/VS16-CI/badge.svg)![Ubuntu 18.04 CI (GCC 7)](https://github.com/lemire/fast_double_parser/workflows/Ubuntu%2018.04%20CI%20(GCC%207)/badge.svg)![VS16-Ninja-CI](https://github.com/lemire/fast_double_parser/workflows/VS16-Ninja-CI/badge.svg)![MSYS2-CI](https://github.com/lemire/fast_double_parser/workflows/MSYS2-CI/badge.svg)![VS16-CLANG-CI](https://github.com/lemire/fast_double_parser/workflows/VS16-CLANG-CI/badge.svg)![MinGW32-CI](https://github.com/lemire/fast_double_parser/workflows/MinGW32-CI/badge.svg)![MinGW64-CI](https://github.com/lemire/fast_double_parser/workflows/MinGW64-CI/badge.svg)![Ubuntu 20.04 CI (GCC 9)](https://github.com/lemire/fast_double_parser/workflows/Ubuntu%2020.04%20CI%20(GCC%209)/badge.svg)[![Build Status](https://api.cirrus-ci.com/github/lemire/fast_double_parser.svg)](https://cirrus-ci.com/github/lemire/fast_double_parser)
# fast_double_parser: 4x faster than strtod
![MSYS2-CI](https://github.com/lemire/fast_double_parser/workflows/MSYS2-CI/badge.svg)[![Ubuntu 22.04](https://github.com/lemire/fast_double_parser/actions/workflows/ubuntu.yml/badge.svg)](https://github.com/lemire/fast_double_parser/actions/workflows/ubuntu.yml)
Unless you need support for [RFC 7159](https://tools.ietf.org/html/rfc7159) (JSON standard), we encourage users to adopt [fast_float](https://github.com/fastfloat/fast_float) library instead. It has more functionality.
Fast function to parse strings containing decimal numbers into double-precision (binary64) floating-point values. That is, given the string "1.0e10", it should return a 64-bit floating-point value equal to 10000000000. We do not sacrifice accuracy. The function will match exactly (down the smallest bit) the result of a standard function like `strtod`.
Fast function to parse ASCII strings containing decimal numbers into double-precision (binary64) floating-point values. That is, given the string "1.0e10", it should return a 64-bit floating-point value equal to 10000000000. We do not sacrifice accuracy. The function will match exactly (down the smallest bit) the result of a standard function like `strtod`.
We support all major compilers: Visual Studio, GNU GCC, LLVM Clang. We require C++11.
The core of this library was ported to Go by Nigel Tao and is now a standard float-parsing routine in Go (`strconv.ParseFloat`).
The core of this library was ported to Go by Nigel Tao and is now a standard float-parsing routine in Go (`strconv.ParseFloat`).
## Reference
- Daniel Lemire, [Number Parsing at a Gigabyte per Second](https://arxiv.org/abs/2101.11408), Software: Practice and Experience 51 (8), 2021.
## Usage
@ -22,13 +30,13 @@ The current API is simple enough:
double x;
char * string = ...
bool isok = fast_double_parser::parse_number(string, &x);
const char * endptr = fast_double_parser::parse_number(string, &x);
```
You must check the value returned (`isok`): if it is `nullptr`, then the function refused to parse the input.
Otherwise, we return a pointer (`const char *`) to the end of the parsed string. The provided
pointer (`string`) should point at the beginning of the number: if you must skip whitespace characters,
it is your responsability to do so.
You must check the value returned (`endptr`): if it is `nullptr`, then the function refused to parse the input.
Otherwise, we return a pointer (`const char *`) to the end of the parsed string. The provided
pointer (`string`) should point at the beginning of the number: if you must skip whitespace characters,
it is your responsibility to do so.
We expect string numbers to follow [RFC 7159](https://tools.ietf.org/html/rfc7159) (JSON standard). In particular,
@ -36,19 +44,22 @@ the parser will reject overly large values that would not fit in binary64. It wi
NaN or infinite values.
It works much like the C standard function `strtod` expect that the parsing is locale-independent. E.g., it will parse 0.5 as 1/2, but it will not parse 0,5 as
1/2 even if you are under a French system. Locale independence is by design (it is not a limitation). Like the standard C functions, it expects that the string
representation of your number ends with a non-number character (e.g., a null character, a space, a colon, etc.). If you wish the specify the end point of the string, as is common in C++, please consider the [fast_float](https://github.com/lemire/fast_float) C++ library instead.
1/2 even if you are under a French system. Locale independence is by design (it is not a limitation). Like the standard C functions, it expects that the string
representation of your number ends with a non-number character (e.g., a null character, a space, a colon, etc.). If you wish the specify the end point of the string, as is common in C++, please consider the [fast_float](https://github.com/lemire/fast_float) C++ library instead.
We assume that the rounding mode is set to nearest, the default setting (`std::fegetround() == FE_TONEAREST`). It is uncommon to have a different setting.
## What if I prefer another API?
The [fast_float](https://github.com/lemire/fast_float) offers an API resembling that of the C++17 `std::from_chars` functions. In particular, you can specify the beginning and the end of the string.
Furthermore [fast_float](https://github.com/lemire/fast_float) supports both 32-bit and 64-bit floating-point numbers. The [fast_float](https://github.com/lemire/fast_float) library is part of Apache Arrow.
Furthermore [fast_float](https://github.com/lemire/fast_float) supports both 32-bit and 64-bit floating-point numbers. The [fast_float](https://github.com/lemire/fast_float) library is part of Apache Arrow, GCC 12, Safari/WebKit and other important systems.
## Why should I expect this function to be faster?
Parsing strings into binary numbers (IEEE 754) is surprisingly difficult. Parsing a single number can take hundreds of instructions and CPU cycles, if not thousands. It is relatively easy to parse numbers faster if you sacrifice accuracy (e.g., tolerate 1 ULP errors), but we are interested in "perfect" parsing.
Instead of trying to solve the general problem, we cover what we believe are the most common scenarios, providing really fast parsing. We fall back on the standard library for the difficult cases. We believe that, in this manner, we achieve the best performance on some of the most important cases.
Instead of trying to solve the general problem, we cover what we believe are the most common scenarios, providing really fast parsing. We fall back on the standard library for the difficult cases. We believe that, in this manner, we achieve the best performance on some of the most important cases.
We have benchmarked our parser on a collection of strings from a sample geojson file (canada.json). Here are some of our results:
@ -62,6 +73,15 @@ We have benchmarked our parser on a collection of strings from a sample geojson
(configuration: Apple clang version 11.0.0, I7-7700K)
We expect string numbers to follow [RFC 7159](https://tools.ietf.org/html/rfc7159). In particular,
the parser will reject overly large values that would not fit in binary64. It will not produce
NaN or infinite values. It will refuse to parse `001` or `0.` as these are invalid number strings as
per the [JSON specification](https://tools.ietf.org/html/rfc7159). Users who prefer a more
lenient C++ parser may consider the [fast_float](https://github.com/lemire/fast_float) C++ library.
The parsing is locale-independent. E.g., it will parse 0.5 as 1/2, but it will not parse 0,5 as
1/2 even if you are under a French system.
## Requirements
@ -70,7 +90,7 @@ If you want to run our benchmarks, you should have
- Windows, Linux or macOS; presumably other systems can be supported as well
- A recent C++ compiler
- A recent cmake (cmake 3.11 or better) is necessary for the benchmarks
- A recent cmake (cmake 3.11 or better) is necessary for the benchmarks
This code falls back on your platform's `strdtod_l` /`_strtod_l` implementation for numbers with a long decimal mantissa (more than 19 digits).
@ -94,7 +114,7 @@ Be mindful that the benchmarks include the abseil library which is not supported
```
$ ./benchmark
$ ./benchmark
parsing random integers in the range [0,1)
@ -118,7 +138,7 @@ double-conv 193.97 MB/s
```
$ ./benchmark benchmarks/data/canada.txt
read 111126 lines
read 111126 lines
=== trial 1 ===
@ -140,14 +160,24 @@ double-conv 243.90 MB/s
## Ports and users
- The algorithm is part of the [LLVM standard libraries](https://github.com/llvm/llvm-project/commit/87c016078ad72c46505461e4ff8bfa04819fe7ba).
- This library has been ported to Go and integrated in the Go standard library.
- It part of the database engine [noisepage](https://github.com/cmu-db/noisepage).
- It is part of the [Microsoft LightGBM machine-learning framework](https://github.com/microsoft/LightGBM).
- It is part of the database engine [noisepage](https://github.com/cmu-db/noisepage).
- The library has been reimplemented in [Google wuffs](https://github.com/google/wuffs/).
- [There is a Julia port](https://github.com/JuliaData/Parsers.jl).
- [There is a Rust port](https://github.com/ezrosent/frawk/tree/master/src/runtime/float_parse).
- [There is a Java port](https://github.com/wrandelshofer/FastDoubleParser).
- [There is a C# port](https://github.com/CarlVerret/csFastFloat).
- [Bazel Central Registry](https://registry.bazel.build/modules/fast_double_parser).
## Credit
Contributions are invited.
This is based on an original idea by Michael Eisel (joint work).
## License
You may use this code under either the Apache License 2.0 or
the Boost Software License 1.0.

View file

@ -20,7 +20,7 @@
#include <vector>
#include "double-conversion/ieee.h"
#include "double-conversion/string-to-double.h"
#include "double-conversion/double-conversion.h"
double findmax_fast_double_parser(const std::vector<std::string>& s) {
double answer = 0;

View file

@ -0,0 +1,3 @@
@PACKAGE_INIT@
include("${CMAKE_CURRENT_LIST_DIR}/fast_double_parser-targets.cmake")

View file

@ -10,25 +10,34 @@
#include <cstdlib>
#include <cstring>
#include <locale.h>
#if (defined(sun) || defined(__sun))
#define FAST_DOUBLE_PARSER_SOLARIS
#endif
#if defined(__CYGWIN__) || defined(__MINGW32__) || defined(__MINGW64__)
#define FAST_DOUBLE_PARSER_CYGWIN
#endif
/**
* Determining whether we should import xlocale.h or not is
* a bit of a nightmare.
*/
#if defined(__CYGWIN__) || defined(__MINGW32__) || defined(__MINGW64__)
// Anything at all that is related to cygwin, msys and so forth will
#if defined(FAST_DOUBLE_PARSER_SOLARIS) || defined(FAST_DOUBLE_PARSER_CYGWIN)
// Anything at all that is related to cygwin, msys, solaris and so forth will
// always use this fallback because we cannot rely on it behaving as normal
// gcc.
#include <locale>
#include <sstream>
// workaround for CYGWIN
double cygwin_strtod_l(const char* start, char** end) {
static inline double cygwin_strtod_l(const char* start, char** end) {
double d;
std::stringstream ss;
ss.imbue(std::locale::classic());
ss << start;
ss >> d;
size_t nread = ss.tellg();
if(ss.fail()) { *end = nullptr; }
if(ss.eof()) { ss.clear(); }
auto nread = ss.tellg();
*end = const_cast<char*>(start) + nread;
return d;
}
@ -55,7 +64,7 @@ double cygwin_strtod_l(const char* start, char** end) {
#endif // __has_include
#endif // defined(__CYGWIN__) || defined(__MINGW32__) || defined(__MINGW64__)
#endif // defined(FAST_DOUBLE_PARSER_SOLARIS) || defined(FAST_DOUBLE_PARSER_CYGWIN)
@ -142,7 +151,7 @@ static inline uint64_t _umul128(uint64_t ab, uint64_t cd, uint64_t *hi) {
// We need a backup on old systems.
// credit: https://stackoverflow.com/questions/28868367/getting-the-high-part-of-64-bit-integer-multiplication
uint64_t Emulate64x64to128(uint64_t& r_hi, const uint64_t x, const uint64_t y) {
really_inline uint64_t Emulate64x64to128(uint64_t& r_hi, const uint64_t x, const uint64_t y) {
const uint64_t x0 = (uint32_t)x, x1 = x >> 32;
const uint64_t y0 = (uint32_t)y, y1 = y >> 32;
const uint64_t p11 = x1 * y1, p01 = x0 * y1;
@ -181,28 +190,26 @@ really_inline value128 full_multiplication(uint64_t value1, uint64_t value2) {
return answer;
}
/* result might be undefined when input_num is zero */
inline int leading_zeroes(uint64_t input_num) {
#ifdef _MSC_VER
unsigned long leading_zero = 0;
// Search the mask data from most significant bit (MSB)
// to least significant bit (LSB) for a set bit (1).
#ifdef _WIN64
if (_BitScanReverse64(&leading_zero, input_num))
return (int)(63 - leading_zero);
else
return 64;
#else
if (_BitScanReverse(&leading_zero, (uint32_t)(input_num >> 32)))
return (int)(63 - (leading_zero + 32));
if (_BitScanReverse(&leading_zero, (uint32_t)input_num))
return (int)(63 - leading_zero);
#endif // _WIN64
#else
return __builtin_clzll(input_num);
#endif // _MSC_VER
}
// Precomputed powers of ten from 10^0 to 10^22. These
// can be represented exactly using the double type.
static const double power_of_ten[] = {
1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9, 1e10, 1e11,
1e12, 1e13, 1e14, 1e15, 1e16, 1e17, 1e18, 1e19, 1e20, 1e21, 1e22};
static inline bool is_integer(char c) {
return (c >= '0' && c <= '9');
// this gets compiled to (uint8_t)(c - '0') <= 9 on all decent compilers
@ -220,13 +227,28 @@ static inline bool is_integer(char c) {
* affect the binary significand.
*/
// The mantissas of powers of ten from -308 to 308, extended out to sixty four
// bits. The array contains the powers of ten approximated
// as a 64-bit mantissa. It goes from 10^FASTFLOAT_SMALLEST_POWER to
// 10^FASTFLOAT_LARGEST_POWER (inclusively).
// The mantissa is truncated, and
// never rounded up. Uses about 5KB.
static const uint64_t mantissa_64[] = {
// Attempts to compute i * 10^(power) exactly; and if "negative" is
// true, negate the result.
// This function will only work in some cases, when it does not work, success is
// set to false. This should work *most of the time* (like 99% of the time).
// We assume that power is in the [FASTFLOAT_SMALLEST_POWER,
// FASTFLOAT_LARGEST_POWER] interval: the caller is responsible for this check.
really_inline double compute_float_64(int64_t power, uint64_t i, bool negative,
bool *success) {
// Precomputed powers of ten from 10^0 to 10^22. These
// can be represented exactly using the double type.
static const double power_of_ten[] = {
1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9, 1e10, 1e11,
1e12, 1e13, 1e14, 1e15, 1e16, 1e17, 1e18, 1e19, 1e20, 1e21, 1e22};
// The mantissas of powers of ten from -308 to 308, extended out to sixty four
// bits. The array contains the powers of ten approximated
// as a 64-bit mantissa. It goes from 10^FASTFLOAT_SMALLEST_POWER to
// 10^FASTFLOAT_LARGEST_POWER (inclusively).
// The mantissa is truncated, and
// never rounded up. Uses about 5KB.
static const uint64_t mantissa_64[] = {
0xa5ced43b7e3e9188, 0xcf42894a5dce35ea,
0x818995ce7aa0e1b2, 0xa1ebfb4219491a1f,
0xca66fa129f9b60a6, 0xfd00b897478238d0,
@ -544,10 +566,10 @@ static const uint64_t mantissa_64[] = {
0xbaa718e68396cffd, 0xe950df20247c83fd,
0x91d28b7416cdd27e, 0xb6472e511c81471d,
0xe3d8f9e563a198e5, 0x8e679c2f5e44ff8f};
// A complement to mantissa_64
// complete to a 128-bit mantissa.
// Uses about 5KB but is rarely accessed.
const uint64_t mantissa_128[] = {
// A complement to mantissa_64
// complete to a 128-bit mantissa.
// Uses about 5KB but is rarely accessed.
static const uint64_t mantissa_128[] = {
0x419ea3bd35385e2d, 0x52064cac828675b9,
0x7343efebd1940993, 0x1014ebe6c5f90bf8,
0xd41a26e077774ef6, 0x8920b098955522b4,
@ -866,16 +888,6 @@ const uint64_t mantissa_128[] = {
0x4cdc331d57fa5441, 0xe0133fe4adf8e952,
0x58180fddd97723a6, 0x570f09eaa7ea7648,};
// Attempts to compute i * 10^(power) exactly; and if "negative" is
// true, negate the result.
// This function will only work in some cases, when it does not work, success is
// set to false. This should work *most of the time* (like 99% of the time).
// We assume that power is in the [FASTFLOAT_SMALLEST_POWER,
// FASTFLOAT_LARGEST_POWER] interval: the caller is responsible for this check.
really_inline double compute_float_64(int64_t power, uint64_t i, bool negative,
bool *success) {
// we start with a fast path
// It was described in
// Clinger WD. How to read floating point numbers accurately.
@ -935,7 +947,7 @@ really_inline double compute_float_64(int64_t power, uint64_t i, bool negative,
// In the slow path, we need to adjust i so that it is > 1<<63 which is always
// possible, except if i == 0, so we handle i == 0 separately.
if(i == 0) {
return 0.0;
return negative ? -0.0 : 0.0;
}
@ -960,7 +972,10 @@ really_inline double compute_float_64(int64_t power, uint64_t i, bool negative,
// For power in (-400,350), we have that
// (((152170 + 65536) * power ) >> 16);
// is equal to
// floor(log(5**power)/log(2)) + power
// floor(log(5**power)/log(2)) + power when power >= 0
// and it is equal to
// ceil(log(5**-power)/log(2)) + power when power < 0
//
//
// The 65536 is (1<<16) and corresponds to
// (65536 * power) >> 16 ---> power
@ -1083,12 +1098,14 @@ really_inline double compute_float_64(int64_t power, uint64_t i, bool negative,
// Return the null pointer on error
static const char * parse_float_strtod(const char *ptr, double *outDouble) {
char *endptr;
#if defined(__CYGWIN__) || defined(__MINGW32__) || defined(__MINGW64__)
// workround for cygwin
#if defined(FAST_DOUBLE_PARSER_SOLARIS) || defined(FAST_DOUBLE_PARSER_CYGWIN)
// workround for cygwin, solaris
*outDouble = cygwin_strtod_l(ptr, &endptr);
#elif defined(_WIN32)
static _locale_t c_locale = _create_locale(LC_ALL, "C");
*outDouble = _strtod_l(ptr, &endptr, c_locale);
#elif defined(__PASE__)
*outDouble = strtod(ptr, &endptr);
#else
static locale_t c_locale = newlocale(LC_ALL_MASK, "C", NULL);
*outDouble = strtod_l(ptr, &endptr, c_locale);
@ -1175,7 +1192,6 @@ really_inline const char * parse_number(const char *p, double *outDouble) {
}
int digit_count =
int(p - start_digits - 1); // used later to guard against overflows
int64_t exp_number = 0; // exponential part
if (('e' == *p) || ('E' == *p)) {
++p;
bool neg_exp = false;
@ -1189,7 +1205,7 @@ really_inline const char * parse_number(const char *p, double *outDouble) {
return nullptr;
}
unsigned char digit = *p - '0';
exp_number = digit;
int64_t exp_number = digit;
p++;
if (is_integer(*p)) {
digit = *p - '0';

View file

@ -0,0 +1,31 @@
def format(number):
upper = number // (1<<64)
lower = number % (1<<64)
print(""+hex(upper)+","+hex(lower)+",")
for q in range(-342,0):
power5 = 5 ** -q
z = 0
while( (1<<z) < power5) :
z += 1
if(q >= -27):
b = z + 127
c = 2 ** b // power5 + 1
format(c)
else:
b = 2 * z + 2 * 64
c = 2 ** b // power5 + 1
# truncate
while(c >= (1<<128)):
c //= 2
format(c)
for q in range(0,308+1):
power5 = 5 ** q
# move the most significant bit in position
while(power5 < (1<<127)):
power5 *= 2
# *truncate*
while(power5 >= (1<<128)):
power5 //= 2
format(power5)

7
tests/bogus.cpp Normal file
View file

@ -0,0 +1,7 @@
#include "fast_double_parser.h"
// To detect issues such as
// https://github.com/lemire/fast_double_parser/issues/42
fast_double_parser::value128 my_full_multiplication(uint64_t value1, uint64_t value2) {
return fast_double_parser::full_multiplication(value1, value2);
}

7
tests/rebogus.cpp Normal file
View file

@ -0,0 +1,7 @@
#include "fast_double_parser.h"
// To detect issues such as
// https://github.com/lemire/fast_double_parser/issues/42
fast_double_parser::value128 my_other_full_multiplication(uint64_t value1, uint64_t value2) {
return fast_double_parser::full_multiplication(value1, value2);
}

View file

@ -75,7 +75,11 @@ void check_string(std::string s) {
printf("fast_double_parser refused to parse %s\n", s.c_str());
throw std::runtime_error("fast_double_parser refused to parse");
}
#ifdef _WIN32
#if defined(FAST_DOUBLE_PARSER_SOLARIS) || defined(FAST_DOUBLE_PARSER_CYGWIN)
// workround for cygwin, solaris
char *endptr;
double d = cygwin_strtod_l(s.data(), &endptr);
#elif defined(_WIN32)
static _locale_t c_locale = _create_locale(LC_ALL, "C");
double d = _strtod_l(s.data(), nullptr, c_locale);
#else
@ -204,13 +208,51 @@ void issue13() {
std::cout << "zero maps to zero" << std::endl;
}
void issue40() {
//https://tools.ietf.org/html/rfc7159
// A fraction part is a decimal point followed by one or more digits.
std::string a = "0.";
double x;
bool ok = fast_double_parser::parse_number(a.c_str(), &x);
if(ok) throw std::runtime_error("We should not parse '0.'");
}
void issue32() {
std::string a = "-0";
double x;
bool ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("could not parse -zero.");
if(x != 0) throw std::runtime_error("-zero does not map to zero.");
std::cout << "0zero maps to zero" << std::endl;
std::cout << "zero maps to zero" << std::endl;
}
void negative_subsubnormal_to_negative_zero() {
std::string a = "-1e-999";
double x;
bool ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("could not parse -1e-999");
if(!std::signbit(x)) throw std::runtime_error("-1e-999 signbit is positive");
if(x != -0) throw std::runtime_error("-1e-999 is not -0");
std::cout << "-1e-999 parses to -0" << std::endl;
}
void issue50_fastpath() {
std::string a = "-0.0";
double x;
bool ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("could not parse -0.0");
if(!std::signbit(x)) throw std::runtime_error("-0.0 signbit is positive");
std::cout << "-0.0 signbit is negative" << std::endl;
}
void issue50_off_fastpath() {
std::string a = "-0.0e-22";
double x;
bool ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("could not parse -0.0-22");
if(!std::signbit(x)) throw std::runtime_error("-0.0-22 signbit is positive");
std::cout << "-0.0-22 signbit is negative" << std::endl;
}
void issue23() {
@ -232,6 +274,27 @@ void issue23_2() {
std::cout << "can parse 5e0012" << std::endl;
}
void issue63() {
std::string a = "1-4-abc";
double x;
const char * ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("1-4-abc");
if(ok != a.c_str() + 1) throw std::runtime_error("does not point at the next number");
if(x != 1) throw std::runtime_error("cannot parse 1.");
std::cout << "1-4-abc" << std::endl;
}
void issue2093() {
std::string a = "0.95000000000000000000";
double x;
const char * ok = fast_double_parser::parse_number(a.c_str(), &x);
if(!ok) throw std::runtime_error("0.95000000000000000000");
if(x != 0.95) throw std::runtime_error("cannot parse 0.95000000000000000000.");
std::cout << "0.95000000000000000000" << std::endl;
}
inline void Assert(bool Assertion) {
if (!Assertion)
throw std::runtime_error("bug");
@ -244,7 +307,11 @@ bool basic_test_64bit(std::string vals, double val) {
std::cerr << " I could not parse " << vals << std::endl;
return false;
}
if(ok != vals.c_str() + vals.size()) throw std::runtime_error("does not point at the end.");
if(ok != vals.c_str() + vals.size()) {
std::cout << "gap is " << (ok - vals.c_str()) << std::endl;
throw std::runtime_error("does not point at the end.");
}
if (std::isnan(val)) {
if (!std::isnan(result_value)) {
std::cerr << "not nan" << result_value << std::endl;
@ -262,6 +329,7 @@ bool basic_test_64bit(std::string vals, double val) {
return true;
}
int main() {
const int evl_method = FLT_EVAL_METHOD;
printf("FLT_EVAL_METHOD = %d\n", evl_method);
@ -271,13 +339,18 @@ int main() {
printf("Aborting further tests.");
return EXIT_SUCCESS;
}
issue2093();
Assert(basic_test_64bit("1090544144181609348835077142190",0x1.b8779f2474dfbp+99));
Assert(basic_test_64bit("4503599627370496.5", 4503599627370496.5));
Assert(basic_test_64bit("4503599627370497.5", 4503599627370497.5));
Assert(basic_test_64bit("0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000044501477170144022721148195934182639518696390927032912960468522194496444440421538910330590478162701758282983178260792422137401728773891892910553144148156412434867599762821265346585071045737627442980259622449029037796981144446145705102663115100318287949527959668236039986479250965780342141637013812613333119898765515451440315261253813266652951306000184917766328660755595837392240989947807556594098101021612198814605258742579179000071675999344145086087205681577915435923018910334964869420614052182892431445797605163650903606514140377217442262561590244668525767372446430075513332450079650686719491377688478005309963967709758965844137894433796621993967316936280457084866613206797017728916080020698679408551343728867675409720757232455434770912461317493580281734466552734375", 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000044501477170144022721148195934182639518696390927032912960468522194496444440421538910330590478162701758282983178260792422137401728773891892910553144148156412434867599762821265346585071045737627442980259622449029037796981144446145705102663115100318287949527959668236039986479250965780342141637013812613333119898765515451440315261253813266652951306000184917766328660755595837392240989947807556594098101021612198814605258742579179000071675999344145086087205681577915435923018910334964869420614052182892431445797605163650903606514140377217442262561590244668525767372446430075513332450079650686719491377688478005309963967709758965844137894433796621993967316936280457084866613206797017728916080020698679408551343728867675409720757232455434770912461317493580281734466552734375));
Assert(basic_test_64bit("0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022250738585072008890245868760858598876504231122409594654935248025624400092282356951787758888037591552642309780950434312085877387158357291821993020294379224223559819827501242041788969571311791082261043971979604000454897391938079198936081525613113376149842043271751033627391549782731594143828136275113838604094249464942286316695429105080201815926642134996606517803095075913058719846423906068637102005108723282784678843631944515866135041223479014792369585208321597621066375401613736583044193603714778355306682834535634005074073040135602968046375918583163124224521599262546494300836851861719422417646455137135420132217031370496583210154654068035397417906022589503023501937519773030945763173210852507299305089761582519159720757232455434770912461317493580281734466552734375", 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022250738585072008890245868760858598876504231122409594654935248025624400092282356951787758888037591552642309780950434312085877387158357291821993020294379224223559819827501242041788969571311791082261043971979604000454897391938079198936081525613113376149842043271751033627391549782731594143828136275113838604094249464942286316695429105080201815926642134996606517803095075913058719846423906068637102005108723282784678843631944515866135041223479014792369585208321597621066375401613736583044193603714778355306682834535634005074073040135602968046375918583163124224521599262546494300836851861719422417646455137135420132217031370496583210154654068035397417906022589503023501937519773030945763173210852507299305089761582519159720757232455434770912461317493580281734466552734375));
issue63();
issue40();
issue23();
issue32();
issue50_fastpath();
issue50_off_fastpath();
issue23_2();
unit_tests();
for (int p = -306; p <= 308; p++) {
@ -321,6 +394,7 @@ int main() {
throw std::runtime_error("fast_double_parser disagrees");
}
}
negative_subsubnormal_to_negative_zero();
std::cout << std::endl;
std::cout << "All ok" << std::endl;
printf("Good!\n");