I'm trying to save and read serialized data to file, but, when reading the file back, it stops at specific entry and begins to read it agin and agin for every upcoming entry. The serialized data I'm writing to file is uncorrupted and i can get every stored value back, but I can't say that about the read data.
For exaple, if I write these keys and values to the file:
"a" with value "a"
"b" with value "b"
"c" with value "c"
"d" with value "d"
"e" with value "e"
...I get this back from the file:
"a" with value "a"
"b" with value "b"
"c" with value "c" //At this entry it would break and start to read the same data
"c" with value "c"
"c" with value "c"
In reality this example would work, because it only happens for some serialized images in "PNG" structures. I'm using "stb_image" library for reading the image data and then serializing it using the "zpp_bits" library. I tested the sizes of serialized and unserialized objects and the results are interesting. The normal size of "PNG" structure is 56 bytes large and when serialized it is 16 bytes large. The buggy "PNG" structure is 56 bytes large and when serialized it is 1054 bytes large, but it looks like it's being written to the file just fine.
the code i would use:
PNG srcImage; //The image struct it will break at (the bug is working only with some images)
unsigned char* buffer = stbi_load("someImage.png", &image.imageWidth, &image.imageHeight, &image.colorChannels, STBI_rgb_alpha);
if (!buffer)
{
std::cout << "Failed to read image: " + path << std::endl;
return 1;
}
image.pixels = reinterpret_cast<char*>(buffer);
DataBuffer srcBuffer;
srcBuffer.serialize("somePath", "someName", srcImage);
srcBuffer.serialize("somePath2", "someName2", std::string("someString")); //Adding another entry for the bug to work
DataManager::writeToFile("someFile.dat", srcBuffer);
DataBuffer dstBuffer;
DataManager::readFromFile("someFile.dat", &dstbuffer);
PNG dstImage = dstBuffer.deserialize<PNG>("somePath", "someName"); //The image is uncorrupted
std::string someString = dstBuffer.deserialize<std::string>("somePath2", "someName2"); //This will return empty string, because there is no entry with the given path and name
the "PNG" struct:
struct PNG
{
int imageWidth, imageHeight, colorChannels;
std::string pixels;
};
the struct I'm storing all serialized data in:
struct DataBuffer
{
std::map<std::string, std::vector<std::byte>> data; //All the serialized data is stored in this map
template<typename T>
void serialize(std::string path, std::string name, T obj)
{
path.append("/");
path.append(name);
serialize(path, obj);
}
template<typename T>
void serialize(std::string fullPath, T obj)
{
if (data.contains(fullPath))
{
std::cout << "Data buffer already contains value with te same path!" << std::endl;
}
try
{
zpp::bits::out out(data[fullPath]);
out(obj).or_throw();
}
catch (const std::exception& error)
{
std::cerr << "Serialization failed with error: " << error.what() << std::endl;
}
catch (...) {
std::cerr << "Serialization failed with unknown error!" << std::endl;
}
}
template<typename T>
T deserialize(std::string path, std::string name)
{
path.append("/");
path.append(name);
return deserialize<T>(path);
}
template<typename T>
T deserialize(std::string fullPath)
{
T obj;
if (!data.contains(fullPath))
{
std::cout << "Data buffer doesn't contain value with the given path!" << std::endl;
return obj;
}
try
{
zpp::bits::in in(data[fullPath]);
in(obj).or_throw();
}
catch (const std::exception& error)
{
std::cerr << "Deserialization failed with error: " << error.what() << std::endl;
}
catch (...) {
std::cerr << "Deserialization failed with unknown error!" << std::endl;
}
return obj;
}
};
the write to file function:
static void DataManager::writeToFile(std::string fileName, DataBuffer dataBuffer)
{
std::ofstream ofs(fileName, std::ios::out | std::ios::binary);
if (!ofs.is_open())
{
std::cout << "Failed to open output file stream!" << std::endl;
return;
}
size_t dataCount = dataBuffer.data.size();
ofs.write((char*)&dataCount, 8);
for (auto const& [key, val] : dataBuffer.data)
{
ofs.write(key.c_str(), 40);
size_t entryCount = val.size();
ofs.write((char*)&entryCount, 8);
for (size_t i = 0; i < val.size(); i++)
{
ofs.write((char*)&val[i], 1);
}
}
ofs.close();
}
the read from file function:
static void DataManager::readFromFile(std::string fileName, DataBuffer* dataBuffer)
{
std::ifstream ifs(fileName, std::ios::in, std::ios::binary);
if (!ifs)
{
std::cout << "File doesnt exist!" << std::endl;
return;
}
if (!ifs.is_open())
{
std::cout << "Failed to open input file stream!" << std::endl;
return;
}
size_t dataCount;
ifs.read((char*)&dataCount, 8);
for (size_t i = 0; i < dataCount; i++)
{
char key[40];
ifs.read(key, 40);
size_t entryCount;
ifs.read((char*)&entryCount, 8);
dataBuffer->data[key].resize(entryCount);
for (size_t j = 0; j < entryCount; j++)
{
ifs.read((char*)&dataBuffer->data[key][j], 1);
}
}
ifs.close();
}
Please don't complain about my code that it is bad. I know it. :D
EDIT
Before i forgot to mention that the image, which causes the ifstream buffer to stuck, is "bed_feet_top.png" in the minecraft texture folder.
I updated my code and I found out that the error occurs when reading the buggy "PNG" structure data, which probably causes the ifstream buffer to stuck at the same entry, but the "PNG" structure data is readable and I can recreate "bed_feet_top.png" image. I have no idea how to get the error message from ifstream. Already tried errno, but it is always 0.
the updated write to file function:
void DataManager::writeToFile(std::string fileName, DataBuffer dataBuffer)
{
std::ofstream ofs(fileName, std::ios::out | std::ios::binary);
if (!ofs.is_open())
{
std::cerr << "Failed to open output file stream!" << std::endl;
return;
}
size_t dataCount = dataBuffer.data.size();
ofs.write((char*)&dataCount, 8);
for (auto const& [key, val] : dataBuffer.data)
{
size_t keySize = key.size();
ofs.write((char*)&keySize, sizeof(keySize));
ofs.write(&key[0], keySize);
size_t entryCount = val.size();
ofs.write((char*)&entryCount, 8);
ofs.write((char*)val.data(), entryCount);
if (!ofs.good()) //ofs.good() never returned false to me
{
std::cerr << "Something went wrong when writing " << key << " to the file!" << std::endl;
break;
}
}
ofs.close();
}
the updated read from file function:
void DataManager::readFromFile(std::string fileName, DataBuffer* dataBuffer)
{
std::ifstream ifs(fileName, std::ios::in, std::ios::binary);
if (!ifs)
{
std::cerr << "File doesnt exist!" << std::endl;
return;
}
if (!ifs.is_open())
{
std::cerr << "Failed to open input file stream!" << std::endl;
return;
}
size_t dataCount;
ifs.read((char*)&dataCount, 8);
for (size_t i = 0; i < dataCount; i++)
{
size_t keySize;
ifs.read((char*)&keySize, sizeof(keySize));
std::string key;
key.resize(keySize);
ifs.read(&key[0], keySize);
size_t entryCount;
ifs.read((char*)&entryCount, 8);
dataBuffer->data[key].resize(entryCount);
ifs.read((char*)dataBuffer->data[key].data(), entryCount);
if (!ifs.good()) //ifs.good() returns false after it reads the buggy "PNG" structure data
{
std::cerr << "Something went wrong when reading " << key << " from the file!" << std::endl;
break;
}
}
ifs.close();
}