Can't import utf8 file with unicode text

Open
#7 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
mysql, php
Domain
data, database

Research direction

Reproduce by editing the bundled tests/test.csv to add Greek or Japanese text and uploading it through the plugin's import dialogue, then compare with the umlaut version that works. Start from the code that reads and parses the uploaded CSV, checking how it handles the encoding and the MySQL connection charset. Done when a UTF-8 file with non-Latin text reaches the field-mapping step. The thread has had no reply in over ten years, so confirm the maintainers still accept the fix before starting.

Written by the indexing model from the issue text.

Description

Omeka 1.5.1 and CsvImport v.1.3.3. Collated utf8 MySQL database. Fresh Omeka install.

If I import the bundled tests/test.csv file in the plugin all works correctly. Modifying this sample data to include umlauts works correctly. Modifying data to include Greek or Japanese text results in the test file not being imported. i.e. I'm returned to the import dialogue without having an opportunity to match fields. When the Japanese or Greek text is replaced with Roman text again the file is properly imported by the plugin once more. Test csv file opens as UTF-8 in BBedit and was saved again as such.

Dominant language
PHP
Stars
20
Forks
32
PR merge metrics
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from omeka/plugin-CsvImport

All issues in omeka/plugin-CsvImport

Similar issues

More PHP issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.