Compression Ratio for Float/Double Arrays
還沒有人認領這個 Issue。
評估
研究方向
The issue includes a standalone JUnit test using Snappy’s primitive-array APIs, but names no repository test file or implementation entry point. Start by tracing Snappy.compress(float[]) and Snappy.compress(double[]) and checking existing primitive-array tests. Determine whether the observed output size is expected for these values or indicates a defect; done means the behavior is explained or reproduced by a focused regression test.
由索引模型根據 Issue 內容生成。
描述
Hi
I'm new to Snappy-Java and wrote a basic test for examining the compression ratio for arrays of int, float and double. I'm finding for small arrays of ~10 elements the compressed bytes are greater than the original arrays. For larger arrays of ~100 elements, int[] compression is around 50% but for float[]/double[] the compressed size is greater than the original size, which surely defeats the purpose of running a compressor.
Maybe I'm doing something wrong, but my test code is pasted below. The results of the test functions is below.
Results:
`Hello snappy-java! Snappy-java is a JNI-based wrapper of Snappy, a fast compresser/decompresser.
compressed length: 87, uncompressed length: 96
compressed int[] bytes: 42, uncompressed bytes: 40
compressed float[] bytes: 42, uncompressed bytes: 40
compressed double[] bytes: 74, uncompressed bytes: 80
compressed int[] bytes: 2208, uncompressed bytes: 4096
compressed float[] bytes: 4101, uncompressed bytes: 4096
compressed double[] bytes: 8197, uncompressed bytes: 8192`
Code:
`package utils.tocompress;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.Random;
import java.util.concurrent.ThreadLocalRandom;
import org.junit.jupiter.api.Test;
import org.xerial.snappy.Snappy;
public class SnappyTest {
/**
* Website example.
*/
@Test
public void test_website_example() {
String input = "Hello snappy-java! Snappy-java is a JNI-based wrapper of Snappy, a fast compresser/decompresser.";
try {
byte[] compressed = Snappy.compress(input.getBytes("UTF-8"));
byte[] uncompressed = Snappy.uncompress(compressed);
String result = new String(uncompressed, "UTF-8");
System.out.println(
"compressed length: " + compressed.length + ", uncompressed length: " + uncompressed.length);
System.out.println(result);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* Arrays of numbers.
*/
@Test
public void test_arrays_of_numbers() {
try {
// int //
int[] ints = new int[] { 1, 111, 65, 888, 99, 45654, 112, 12, 45, 10 };
byte[] compressed_ints = Snappy.compress(ints);
int[] uncompressed_ints = Snappy.uncompressIntArray(compressed_ints);
boolean isSame = isSameArray(ints, uncompressed_ints);
if (!isSame) {
System.out.println("NOT SAME");
}
System.out.println(
"compressed int[] bytes: " + compressed_ints.length + ", uncompressed bytes: " + (uncompressed_ints.length*Integer.BYTES));
// float //
float[] floats = new float[] { 1.1f, 111.22f, 65.65f, 888.8f, 99.01f, 45654.12f, 112.112f, 12.1f, 45.54f, 10.1f };
byte[] compressed_floats = Snappy.compress(floats);
float[] uncompressed_floats = Snappy.uncompressFloatArray(compressed_floats);
isSame = isSameArray(floats, uncompressed_floats, 0.001f);
if (!isSame) {
System.out.println("NOT SAME");
}
System.out.println(
"compressed float[] bytes: " + compressed_floats.length + ", uncompressed bytes: " + (uncompressed_floats.length*Float.BYTES));
// double //
double[] doubles = new double[] { 1.1, 111.22, 65.65, 888.8, 99.01, 45654.12, 112.112, 12.1, 45.54, 10.1 };
byte[] compressed_doubles = Snappy.compress(doubles);
double[] uncompressed_doubles = Snappy.uncompressDoubleArray(compressed_doubles);
isSame = isSameArray(doubles, uncompressed_doubles, 0.001);
if (!isSame) {
System.out.println("NOT SAME");
}
System.out.println(
"compressed double[] bytes: " + compressed_doubles.length + ", uncompressed bytes: " + (uncompressed_doubles.length*Double.BYTES));
// int //
int numValues = 1024;
int min = 0;
int max = 100;
int[] ints_random = new int[numValues];
for (int i=0; i<numValues; i++) {
int randomNum = ThreadLocalRandom.current().nextInt(min, max + 1);
ints_random[i] = randomNum;
}
compressed_ints = Snappy.compress(ints_random);
uncompressed_ints = Snappy.uncompressIntArray(compressed_ints);
isSame = isSameArray(ints_random, uncompressed_ints);
if (!isSame) {
System.out.println("NOT SAME");
}
System.out.println(
"compressed int[] bytes: " + compressed_ints.length + ", uncompressed bytes: " + (ints_random.length*Integer.BYTES));
// float //
Random r = new Random();
float[] floats_random = new float[numValues];
for (int i=0; i<numValues; i++) {
float random = min + r.nextFloat() * (max - min);
floats_random[i] = random;
}
compressed_floats = Snappy.compress(floats_random);
uncompressed_floats = Snappy.uncompressFloatArray(compressed_floats);
isSame = isSameArray(floats_random, uncompressed_floats, 0.001f);
if (!isSame) {
System.out.println("NOT SAME");
}
System.out.println(
"compressed float[] bytes: " + compressed_floats.length + ", uncompressed bytes: " + (floats_random.length*Float.BYTES));
// double //
double[] doubles_random = new double[numValues];
for (int i=0; i<numValues; i++) {
double random = min + r.nextDouble() * (max - min);
doubles_random[i] = random;
}
byte[] compressed_doubles_r = Snappy.compress(doubles_random);
double[] uncompressed_doubles_r = Snappy.uncompressDoubleArray(compressed_doubles_r);
isSame = isSameArray(doubles_random, uncompressed_doubles_r, 0.001);
if (!isSame) {
System.out.println("NOT SAME");
}
System.out.println(
"compressed double[] bytes: " + compressed_doubles_r.length + ", uncompressed bytes: " + (doubles_random.length*Double.BYTES));
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public boolean isSameArray(int[] array1, int[] array2) {
if (array1.length != array2.length) {
return false;
}
for (int i=0; i<array1.length; i++) {
if (array1[i] != array2[i]) {
return false;
}
}
return true;
}
public boolean isSameArray(float[] array1, float[] array2, float tol) {
if (array1.length != array2.length) {
return false;
}
for (int i=0; i<array1.length; i++) {
if (Math.abs(array1[i] - array2[i]) > tol) {
return false;
}
}
return true;
}
public boolean isSameArray(double[] array1, double[] array2, double tol) {
if (array1.length != array2.length) {
return false;
}
for (int i=0; i<array1.length; i++) {
if (Math.abs(array1[i] - array2[i]) > tol) {
return false;
}
}
return true;
}
} // class SnappyTest
`
- 主要語言
- Java
- 星號
- 1.1k
- 分支
- 261
- 平均合併
- 2 分鐘
- 30 天內合併 PR
- 2
環境準備
這個專案沒有提供開發容器、Dockerfile 或貢獻指南,環境需要你自己搭建:先看它的 README,通用步驟見我們的新手貢獻指南。
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
xerial/snappy-java 的其他 Issue
-
難度 4/5 3-5 天 新手友好度 35/100
xerial/snappy-java#658 · 3 則留言 · 1 個 reaction ·
-
難度 3/5 1-2 天 新手友好度 56/100
xerial/snappy-java#607 ·
-
難度 4/5 3-5 天 新手友好度 45/100
xerial/snappy-java#604 · 1 則留言 ·
-
難度 4/5 3-5 天 新手友好度 45/100
xerial/snappy-java#603 · 3 則留言 ·
-
Snappy error on Mac OS Ventura: "no native library is found for os.name=Mac and os.arch=aarch64"未關閉
難度 4/5 3-5 天 新手友好度 35/100
xerial/snappy-java#573 · 1 個 reaction ·
查看 xerial/snappy-java 的全部 Issue
相似的 Issue
-
hermes-queue
難度 1/5 1-3 小時 新手友好度 85/100
chrisalunlloyd2-sudo/mindpalace#403 ·
維護者通常 1 天內回覆
-
triage
難度 2/5 1-3 小時 新手友好度 72/100
xerial/sqlite-jdbc#1515 ·
維護者通常 1 天內回覆
-
Area/Configuration Type/Improvement
難度 2/5 1-3 小時 新手友好度 68/100
ballerina-platform/static-code-analysis-tool#217 ·
維護者通常 1 天內回覆
-
SequenceEncoder: a value ending with a backslash swallows the tokens after it (“No state for trait”)未關閉
難度 2/5 1-3 小時 新手友好度 80/100
vassalengine/vassal#15337 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 76/100
eclipse-dirigible/dirigible#8636 · 1 則留言 ·
維護者通常 1 天內回覆