In [1]:
Copied!
import os
os.environ["SPARSE_BACKEND"] = "Finch"
import sparse
import numpy as np
import os
os.environ["SPARSE_BACKEND"] = "Finch"
import sparse
import numpy as np
/home/docs/checkouts/readthedocs.org/user_builds/sparse-nd/checkouts/latest/sparse/__init__.py:24: SparseFutureWarning: Changing back-ends is a development feature, please do not rely on it in production. warnings.warn(
[juliapkg] Found dependencies: /home/docs/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/lib/python3.12/site-packages/juliacall/juliapkg.json [juliapkg] Found dependencies: /home/docs/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/lib/python3.12/site-packages/finch/juliapkg.json [juliapkg] Found dependencies: /home/docs/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/lib/python3.12/site-packages/juliapkg/juliapkg.json [juliapkg] Locating Julia 1.10.3 - 1.11 [juliapkg] Querying Julia versions from https://julialang-s3.julialang.org/bin/versions.json [juliapkg] WARNING: About to install Julia 1.11.9 to /home/docs/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/julia_env/pyjuliapkg/install. [juliapkg] If you use juliapkg in more than one environment, you are likely to [juliapkg] have Julia installed in multiple locations. It is recommended to [juliapkg] install JuliaUp (https://github.com/JuliaLang/juliaup) or Julia [juliapkg] (https://julialang.org/downloads) yourself. [juliapkg] Downloading Julia from https://julialang-s3.julialang.org/bin/linux/x64/1.11/julia-1.11.9-linux-x86_64.tar.gz
download complete [juliapkg] Verifying download [juliapkg] Installing Julia 1.11.9 to /home/docs/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/julia_env/pyjuliapkg/install
[juliapkg] Using Julia 1.11.9 at /home/docs/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/julia_env/pyjuliapkg/install/bin/julia
[juliapkg] Finding libjulia:
| using Libdl
| print(abspath(Libdl.dlpath("libjulia")))
[juliapkg] Using Julia project at /home/docs/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/julia_env
[juliapkg] Writing Project.toml:
| [deps]
| PythonCall = "6099a3de-0909-46bc-b1f4-468b9a2dfc0d"
| OpenSSL_jll = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
| Finch = "9177782c-1635-4eb9-9bfb-d9dfa25e6bce"
| HDF5 = "f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f"
| NPZ = "15e1cf62-19b3-5cfa-8e77-841668bca605"
| TensorMarket = "8b7d4fe7-0b45-4d0d-9dd8-5cc9b23b4b77"
|
| [compat]
| PythonCall = "=0.9.34"
| OpenSSL_jll = "~3.0"
| Finch = "^1.2.10"
| HDF5 = "^0.17.2"
| NPZ = "^0.4.3"
| TensorMarket = "^0.2"
[juliapkg] Installing packages:
| import Pkg
| Pkg.Registry.update()
| Pkg.add([
| Pkg.PackageSpec(name="PythonCall", uuid="6099a3de-0909-46bc-b1f4-468b9a2dfc0d"),
| Pkg.PackageSpec(name="OpenSSL_jll", uuid="458c3c95-2e84-50aa-8efc-19380b2a3a95"),
| Pkg.PackageSpec(name="Finch", uuid="9177782c-1635-4eb9-9bfb-d9dfa25e6bce"),
| Pkg.PackageSpec(name="HDF5", uuid="f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f"),
| Pkg.PackageSpec(name="NPZ", uuid="15e1cf62-19b3-5cfa-8e77-841668bca605"),
| Pkg.PackageSpec(name="TensorMarket", uuid="8b7d4fe7-0b45-4d0d-9dd8-5cc9b23b4b77"),
| ])
| Pkg.resolve()
| Pkg.precompile()
Installing known registries into `~/.julia`
Added `General` registry to ~/.julia/registries
Updating registry at `~/.julia/registries/General.toml` Resolving package versions...
Installed libaec_jll ────────────────── v1.1.7+0 Installed Scratch ───────────────────── v1.3.0 Installed MPIABI_jll ────────────────── v0.1.5+0 Installed dlfcn_win32_jll ───────────── v1.4.2+0 Installed MPICH_jll ─────────────────── v5.0.1+0 Installed AutoHashEquals ────────────── v2.2.0 Installed MicroMamba ────────────────── v0.1.15 Installed IrrationalConstants ───────── v0.2.6 Installed JSON ──────────────────────── v1.6.0 Installed aws_c_common_jll ──────────── v0.12.6+0 Installed XML2_jll ──────────────────── v2.13.9+0
Installed Xorg_libpciaccess_jll ─────── v0.19.0+0 Installed Rmath ─────────────────────── v0.9.0 Installed HypergeometricFunctions ───── v0.3.28 Installed aws_c_auth_jll ────────────── v0.9.6+0 Installed Preferences ───────────────── v1.5.2 Installed PythonCall ────────────────── v0.9.34 Installed TableTraits ───────────────── v1.0.1 Installed SpecialFunctions ──────────── v2.8.0 Installed NPZ ───────────────────────── v0.4.3 Installed StatsFuns ─────────────────── v1.5.2 Installed IterTools ─────────────────── v1.10.0 Installed aws_c_compression_jll ─────── v0.3.2+0 Installed Tables ────────────────────── v1.12.1 Installed PtrArrays ─────────────────── v1.4.0
Installed OpenSSL_jll ───────────────── v3.0.20+0 Installed PDMats ────────────────────── v0.11.37 Installed DataAPI ───────────────────── v1.16.0 Installed Hwloc_jll ─────────────────── v2.13.0+1 Installed Parsers ───────────────────── v2.8.4 Installed MicrosoftMPI_jll ──────────── v10.1.4+3 Installed aws_c_http_jll ────────────── v0.10.13+0 Installed ZipFile ───────────────────── v0.10.1 Installed SyntaxInterface ───────────── v0.2.1 Installed Pidfile ───────────────────── v1.3.0 Installed PrecompileTools ───────────── v1.2.1 Installed IteratorInterfaceExtensions ─ v1.0.0 Installed OpenMPI_jll ───────────────── v5.0.11+0
Installed JLLWrappers ───────────────── v1.8.0 Installed OrderedCollections ────────── v1.8.2 Installed aws_c_io_jll ──────────────── v0.26.3+0 Installed micromamba_jll ────────────── v2.3.1+0 Installed DataValueInterfaces ───────── v1.0.0 Installed Libiconv_jll ──────────────── v1.18.0+0 Installed AbstractTrees ─────────────── v0.4.5
Installed aws_checksums_jll ─────────── v0.2.10+0 Installed FillArrays ────────────────── v1.16.0 Installed RewriteTools ──────────────── v0.2.4 Installed TensorMarket ──────────────── v0.2.2 Installed FileIO ────────────────────── v1.19.0 Installed QuadGK ────────────────────── v2.11.3 Installed HDF5 ──────────────────────── v0.17.3 Installed Finch ─────────────────────── v1.2.12 Installed Reexport ──────────────────── v1.2.2
Installed DataStructures ────────────── v0.19.5 Installed MPIPreferences ────────────── v0.1.12 Installed Rmath_jll ─────────────────── v0.5.1+0 Installed Statistics ────────────────── v1.11.1 Installed LogExpFunctions ───────────── v0.3.29 Installed mpif_jll ──────────────────── v0.1.7+0 Installed AliasTables ───────────────── v1.1.3 Installed Requires ──────────────────── v1.3.1 Installed MPItrampoline_jll ─────────── v5.5.6+0 Installed StatsAPI ──────────────────── v1.8.0 Installed OpenSpecFun_jll ───────────── v0.5.6+0
Installed aws_c_sdkutils_jll ────────── v0.2.4+1 Installed UnsafePointers ────────────── v1.0.0 Installed MacroTools ────────────────── v0.5.16 Installed Distributions ─────────────── v0.25.125 Installed Compat ────────────────────── v4.18.1 Installed UnsafeAtomics ─────────────── v0.3.1 Installed CondaPkg ──────────────────── v0.2.36 Installed HDF5_jll ──────────────────── v2.1.2+0 Installed Missings ──────────────────── v1.2.0 Installed StatsBase ─────────────────── v0.34.11 Installed SortingAlgorithms ─────────── v1.2.2 Installed DocStringExtensions ───────── v0.9.5 Installed StructUtils ───────────────── v2.8.2 Installed aws_c_cal_jll ─────────────── v0.9.13+0
Installed aws_c_s3_jll ──────────────── v0.11.5+0 Installed pixi_jll ──────────────────── v0.63.2+0 Installed s2n_tls_jll ───────────────── v1.7.3+0
Updating `~/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/julia_env/Project.toml`
[9177782c] + Finch v1.2.12
[f67ccb44] + HDF5 v0.17.3
[15e1cf62] + NPZ v0.4.3
[6099a3de] + PythonCall v0.9.34
[8b7d4fe7] + TensorMarket v0.2.2
⌅ [458c3c95] + OpenSSL_jll v3.0.20+0
Updating `~/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/julia_env/Manifest.toml`
[1520ce14] + AbstractTrees v0.4.5
[66dad0bd] + AliasTables v1.1.3
[15f4f7f2] + AutoHashEquals v2.2.0
[34da2185] + Compat v4.18.1
[992eb4ea] + CondaPkg v0.2.36
[9a962f9c] + DataAPI v1.16.0
[864edb3b] + DataStructures v0.19.5
[e2d170a0] + DataValueInterfaces v1.0.0
[31c24e10] + Distributions v0.25.125
[ffbed154] + DocStringExtensions v0.9.5
[5789e2e9] + FileIO v1.19.0
[1a297f60] + FillArrays v1.16.0
[9177782c] + Finch v1.2.12
[f67ccb44] + HDF5 v0.17.3
[34004b35] + HypergeometricFunctions v0.3.28
[92d709cd] + IrrationalConstants v0.2.6
[c8e1da08] + IterTools v1.10.0
[82899510] + IteratorInterfaceExtensions v1.0.0
[692b3bcd] + JLLWrappers v1.8.0
[682c06a0] + JSON v1.6.0
⌅ [2ab3a3ac] + LogExpFunctions v0.3.29
[3da0fdf6] + MPIPreferences v0.1.12
[1914dd2f] + MacroTools v0.5.16
[0b3b1443] + MicroMamba v0.1.15
[e1d29d7a] + Missings v1.2.0
[15e1cf62] + NPZ v0.4.3
[bac558e1] + OrderedCollections v1.8.2
[90014a1f] + PDMats v0.11.37
[69de0a69] + Parsers v2.8.4
[fa939f87] + Pidfile v1.3.0
⌅ [aea7be01] + PrecompileTools v1.2.1
[21216c6a] + Preferences v1.5.2
[43287f4e] + PtrArrays v1.4.0
[6099a3de] + PythonCall v0.9.34
[1fd47b50] + QuadGK v2.11.3
[189a3867] + Reexport v1.2.2
[ae029012] + Requires v1.3.1
[5969e224] + RewriteTools v0.2.4
[79098fc4] + Rmath v0.9.0
[6c6a2e73] + Scratch v1.3.0
[a2af1166] + SortingAlgorithms v1.2.2
[276daf66] + SpecialFunctions v2.8.0
[10745b16] + Statistics v1.11.1
[82ae8749] + StatsAPI v1.8.0
[2913bbd2] + StatsBase v0.34.11
⌅ [4c63d2b9] + StatsFuns v1.5.2
[ec057cc2] + StructUtils v2.8.2
[b33eeca9] + SyntaxInterface v0.2.1
[3783bdb8] + TableTraits v1.0.1
[bd369af6] + Tables v1.12.1
[8b7d4fe7] + TensorMarket v0.2.2
[013be700] + UnsafeAtomics v0.3.1
[e17b2a0c] + UnsafePointers v1.0.0
[a5390f91] + ZipFile v0.10.1
[0234f1f7] + HDF5_jll v2.1.2+0
[e33a78d0] + Hwloc_jll v2.13.0+1
[94ce4f54] + Libiconv_jll v1.18.0+0
[b5ada748] + MPIABI_jll v0.1.5+0
[7cb0a576] + MPICH_jll v5.0.1+0
[f1f71cc9] + MPItrampoline_jll v5.5.6+0
[9237b28f] + MicrosoftMPI_jll v10.1.4+3
[fe0851c0] + OpenMPI_jll v5.0.11+0
⌅ [458c3c95] + OpenSSL_jll v3.0.20+0
[efe28fd5] + OpenSpecFun_jll v0.5.6+0
[f50d1b31] + Rmath_jll v0.5.1+0
⌅ [02c8fc9c] + XML2_jll v2.13.9+0
[a65dc6b1] + Xorg_libpciaccess_jll v0.19.0+0
⌅ [2b3700d1] + aws_c_auth_jll v0.9.6+0
[70f11efc] + aws_c_cal_jll v0.9.13+0
[73048d1d] + aws_c_common_jll v0.12.6+0
[73a04cd5] + aws_c_compression_jll v0.3.2+0
[3254fc65] + aws_c_http_jll v0.10.13+0
[13c41daa] + aws_c_io_jll v0.26.3+0
⌅ [bd1f34fb] + aws_c_s3_jll v0.11.5+0
[1282aa60] + aws_c_sdkutils_jll v0.2.4+1
[b2a88e68] + aws_checksums_jll v0.2.10+0
[c4b69c83] + dlfcn_win32_jll v1.4.2+0
[477f73a3] + libaec_jll v1.1.7+0
[f8abcde7] + micromamba_jll v2.3.1+0
[9aeb927a] + mpif_jll v0.1.7+0
[4d7b5844] + pixi_jll v0.63.2+0
[cddc5d3d] + s2n_tls_jll v1.7.3+0
[0dad84c5] + ArgTools v1.1.2
[56f22d72] + Artifacts v1.11.0
[2a0f44e3] + Base64 v1.11.0
[ade2ca70] + Dates v1.11.0
[f43a241f] + Downloads v1.6.0
[7b1f6079] + FileWatching v1.11.0
[b77e0a4c] + InteractiveUtils v1.11.0
[4af54fe1] + LazyArtifacts v1.11.0
[b27032c2] + LibCURL v0.6.4
[76f85450] + LibGit2 v1.11.0
[8f399da3] + Libdl v1.11.0
[37e2e46d] + LinearAlgebra v1.11.0
[56ddb016] + Logging v1.11.0
[d6f4376e] + Markdown v1.11.0
[a63ad114] + Mmap v1.11.0
[ca575930] + NetworkOptions v1.2.0
[44cfe95a] + Pkg v1.11.0
[de0858da] + Printf v1.11.0
[9a3f8284] + Random v1.11.0
[ea8e919c] + SHA v0.7.0
[9e88b42a] + Serialization v1.11.0
[2f01184e] + SparseArrays v1.11.0
[4607b0f0] + SuiteSparse
[fa267f1f] + TOML v1.0.3
[a4e569a6] + Tar v1.10.0
[8dfed614] + Test v1.11.0
[cf7118a7] + UUIDs v1.11.0
[4ec0a83e] + Unicode v1.11.0
[e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
[deac9b47] + LibCURL_jll v8.6.0+0
[e37daf67] + LibGit2_jll v1.7.2+0
[29816b5a] + LibSSH2_jll v1.11.0+1
[c8ffd9c3] + MbedTLS_jll v2.28.6+0
[14a3606d] + MozillaCACerts_jll v2023.12.12
[4536629a] + OpenBLAS_jll v0.3.27+1
[05823500] + OpenLibm_jll v0.8.5+0
[bea87d4a] + SuiteSparse_jll v7.7.0+0
[83775a58] + Zlib_jll v1.2.13+1
[8e850b90] + libblastrampoline_jll v5.11.0+0
[8e850ede] + nghttp2_jll v1.59.0+0
[3f19e933] + p7zip_jll v17.4.0+2
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling project...
666.7 ms ✓ TensorMarket
802.2 ms ✓ AutoHashEquals
957.6 ms ✓ OrderedCollections
545.5 ms ✓ DataValueInterfaces
535.7 ms ✓ Reexport
924.3 ms ✓ DocStringExtensions 1719.4 ms ✓ FillArrays
1021.6 ms ✓ AbstractTrees
766.4 ms ✓ Statistics
733.1 ms ✓ UnsafePointers
367.8 ms ✓ IteratorInterfaceExtensions
499.9 ms ✓ DataAPI
580.7 ms ✓ StatsAPI
2704.9 ms ✓ IrrationalConstants
920.1 ms ✓ IterTools
568.9 ms ✓ SyntaxInterface
559.9 ms ✓ PtrArrays
753.5 ms ✓ Requires 4196.4 ms ✓ UnsafeAtomics
1073.2 ms ✓ ZipFile
1548.7 ms ✓ StructUtils
751.0 ms ✓ Scratch
731.9 ms ✓ Pidfile
1071.8 ms ✓ SuiteSparse 5156.1 ms ✓ MacroTools
1016.4 ms ✓ Compat
1162.1 ms ✓ Preferences
1333.0 ms ✓ FillArrays → FillArraysSparseArraysExt 1056.5 ms ✓ Statistics → SparseArraysExt
610.8 ms ✓ TableTraits
781.0 ms ✓ FillArrays → FillArraysStatisticsExt
3347.5 ms ✓ DataStructures
691.6 ms ✓ Missings
1062.9 ms ✓ LogExpFunctions
738.5 ms ✓ AliasTables
935.9 ms ✓ RewriteTools
643.8 ms ✓ Compat → CompatLinearAlgebraExt
822.3 ms ✓ MPIPreferences 1770.5 ms ✓ PDMats
908.7 ms ✓ PrecompileTools
847.0 ms ✓ JLLWrappers
829.5 ms ✓ SortingAlgorithms
1629.4 ms ✓ Tables
992.0 ms ✓ FillArrays → FillArraysPDMatsExt
1932.5 ms ✓ QuadGK
1020.3 ms ✓ OpenSSL_jll
1124.8 ms ✓ Rmath_jll
823.8 ms ✓ MicrosoftMPI_jll
838.9 ms ✓ dlfcn_win32_jll
1044.8 ms ✓ aws_c_common_jll
10736.1 ms ✓ FileIO
1049.9 ms ✓ Libiconv_jll
2101.7 ms ✓ MPItrampoline_jll
2219.2 ms ✓ micromamba_jll
989.3 ms ✓ Xorg_libpciaccess_jll
942.7 ms ✓ OpenSpecFun_jll
970.5 ms ✓ libaec_jll
2250.8 ms ✓ pixi_jll
972.1 ms ✓ s2n_tls_jll
766.9 ms ✓ StructUtils → StructUtilsTablesExt
1590.1 ms ✓ Rmath
4159.8 ms ✓ StatsBase 1225.7 ms ✓ aws_checksums_jll
1025.9 ms ✓ aws_c_cal_jll
1181.7 ms ✓ aws_c_sdkutils_jll
1013.9 ms ✓ aws_c_compression_jll
1190.5 ms ✓ XML2_jll
3051.6 ms ✓ NPZ
5074.7 ms ✓ MicroMamba
4734.7 ms ✓ SpecialFunctions
970.1 ms ✓ PDMats → StatsBaseExt
1169.0 ms ✓ aws_c_io_jll 1022.5 ms ✓ Hwloc_jll
1185.8 ms ✓ aws_c_http_jll
1831.5 ms ✓ HypergeometricFunctions
2283.0 ms ✓ MPIABI_jll 1849.9 ms ✓ OpenMPI_jll
1048.1 ms ✓ aws_c_auth_jll
1451.9 ms ✓ MPICH_jll
1171.1 ms ✓ aws_c_s3_jll
3159.8 ms ✓ StatsFuns
2176.5 ms ✓ mpif_jll
2419.9 ms ✓ HDF5_jll
36245.3 ms ✓ Parsers
9842.2 ms ✓ Distributions
2221.2 ms ✓ Distributions → DistributionsTestExt
9165.8 ms ✓ JSON
16232.6 ms ✓ HDF5
8603.9 ms ✓ CondaPkg
28488.7 ms ✓ PythonCall
274212.2 ms ✓ Finch
4913.5 ms ✓ Finch → HDF5Ext
5160.0 ms ✓ Finch → SparseArraysExt
5837.2 ms ✓ Finch → NPZExt
2395.2 ms ✓ Finch → TensorMarketExt
95 dependencies successfully precompiled in 347 seconds. 32 already precompiled.
3 dependencies had output during precompilation:
┌ Finch
│ [ Info: Running enhanced Finch.jl precompilation... (to disable, run `using Preferences; Preferences.set_preferences!("Finch", "precompile_workload" => false; force=true)`). See https://julialang.github.io/PrecompileTools.jl/stable/#Package-developers:-reducing-the-cost-of-precompilation-during-development for more.
│ [ Info: Precompiling common tensor formats...
│ [ Info: Precompiling common tensor formats... (estimated: 0 minutes and 3 seconds)
│ [ Info: Precompiling common tensor operations...
│
│ [ Info: Precompiling common tensor operations... (estimated: 2 minutes and 36 seconds)
│
│ [ Info: Precompiling common tensor operations... (estimated: 1 minutes and 33 seconds)
│
│ [ Info: Precompiling common tensor operations... (estimated: 1 minutes and 44 seconds)
│
│ [ Info: Precompiling common tensor operations... (estimated: 1 minutes and 15 seconds)
│
│ [ Info: Precompiling common tensor operations... (estimated: 0 minutes and 58 seconds)
│
│ [ Info: Precompiling common tensor operations... (estimated: 0 minutes and 41 seconds)
│
│ [ Info: Precompiling common tensor operations... (estimated: 0 minutes and 28 seconds)
│
│ [ Info: Precompiling common tensor operations... (estimated: 0 minutes and 18 seconds)
│
│ [ Info: Precompiling common tensor operations... (estimated: 0 minutes and 9 seconds)
│
│ [ Info: Done!
└
┌ CondaPkg
│ Downloading artifact: pixi
└
┌ MicroMamba
│ Downloading artifact: micromamba
└
No Changes to `~/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/julia_env/Project.toml` No Changes to `~/checkouts/readthedocs.org/user_builds/sparse-nd/envs/latest/julia_env/Manifest.toml`
Detected IPython. Loading juliacall extension. See https://juliapy.github.io/PythonCall.jl/stable/compat/#IPython
Perform Operations¶
Let's create two arrays.
In [2]:
Copied!
rng = np.random.default_rng(42) # Seed for reproducibility
a = sparse.random((3, 3), density=1 / 6, random_state=rng)
b = sparse.random((3, 3), density=1 / 6, random_state=rng)
rng = np.random.default_rng(42) # Seed for reproducibility
a = sparse.random((3, 3), density=1 / 6, random_state=rng)
b = sparse.random((3, 3), density=1 / 6, random_state=rng)
Now let's matrix multiply them.
In [3]:
Copied!
c = a @ b
c = a @ b
And view the result as a (dense) NumPy array.
In [4]:
Copied!
c_dense = c.todense()
c_dense = c.todense()
Now let's do the same for other formats, and compare the results.
In [5]:
Copied!
for format in ["coo", "csr", "csc", "dense"]:
af = sparse.asarray(a, format=format)
bf = sparse.asarray(b, format=format)
cf = af @ bf
np.testing.assert_array_equal(c_dense, cf.todense())
for format in ["coo", "csr", "csc", "dense"]:
af = sparse.asarray(a, format=format)
bf = sparse.asarray(b, format=format)
cf = af @ bf
np.testing.assert_array_equal(c_dense, cf.todense())