Mixed drops insert audio as an image

Open
#23,388 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
56/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
android, kotlin
Domain
mobile

Research direction

The issue includes a reproduction test in AnkiDroid/src/test/java/com/ichi2/anki/noteeditor/NoteEditorDropMixedMediaTest.kt and points to content handling in the note editor. Start by tracing how NoteEditorFragment receives mixed drag and drop content, then run the added test with both image and audio items. Done means the image is inserted as an image and the audio as [sound:...]; the issue notes this predates the referenced pull request and may need a separate issue.

Written by the indexing model from the issue text.

Description

Needs Triage

Assisted-by: Claude Opus 5.5 max

Subject: [EXISTING] Mixed drops insert audio as an image

Dropping an image and an audio file together inserts the audio as an image tag; with Paste as PNG on it is dropped.

⚠️ This issue existed before the PR and should likely be split off into a separate issue.

---
diff --git a/AnkiDroid/src/test/java/com/ichi2/anki/noteeditor/NoteEditorDropMixedMediaTest.kt b/AnkiDroid/src/test/java/com/ichi2/anki/noteeditor/NoteEditorDropMixedMediaTest.kt
new file mode 100644
index 0000000000..85c8449237
--- /dev/null
+++ b/AnkiDroid/src/test/java/com/ichi2/anki/noteeditor/NoteEditorDropMixedMediaTest.kt
@@ -0,0 +1,75 @@
+// SPDX-License-Identifier: GPL-3.0-or-later
+
+package com.ichi2.anki.noteeditor
+
+import android.content.ClipData
+import android.content.ClipDescription
+import android.content.ContentProvider
+import android.database.MatrixCursor
+import android.graphics.Bitmap
+import android.provider.OpenableColumns
+import androidx.core.graphics.createBitmap
+import androidx.core.net.toUri
+import androidx.core.view.ContentInfoCompat
+import androidx.core.view.ViewCompat
+import androidx.test.ext.junit.runners.AndroidJUnit4
+import com.ichi2.anki.NoteEditorActivity
+import com.ichi2.anki.RobolectricTest
+import com.ichi2.anki.common.destinations.NoteEditorDestination
+import com.ichi2.anki.common.utils.ext.pasteImagesAsPng
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.mockito.kotlin.mock
+import org.mockito.kotlin.whenever
+import org.robolectric.Shadows.shadowOf
+import org.robolectric.annotation.GraphicsMode
+import org.robolectric.shadows.ShadowContentResolver
+import java.io.ByteArrayOutputStream
+import kotlin.test.assertContains
+
+@RunWith(AndroidJUnit4::class)
+@GraphicsMode(GraphicsMode.Mode.NATIVE)
+class NoteEditorDropMixedMediaTest : RobolectricTest() {
+    override fun getCollectionStorageMode() = CollectionStorageMode.ON_DISK
+
+    @Test
+    fun `dropping an image with audio inserts the audio as sound`() =
+        runTest {
+            col.config.pasteImagesAsPng = false
+            val note = addBasicNote("Front", "Back")
+            val editor =
+                startRegularActivity<NoteEditorActivity>(
+                    NoteEditorDestination.EditNoteFromPreviewer(note.firstCard().id).toIntent(targetContext),
+                ).getNoteEditorFragment()
+            val image = "content://files/photo.jpg".toUri()
+            val audio = "content://files/song.mp3".toUri()
+            val imageData =
+                ByteArrayOutputStream().use {
+                    createBitmap(1, 1).compress(Bitmap.CompressFormat.JPEG, 100, it)
+                    it.toByteArray()
+                }
+            val provider = mock<ContentProvider>()
+            whenever(provider.query(image, null, null, null, null))
+                .thenReturn(MatrixCursor(arrayOf(OpenableColumns.DISPLAY_NAME)).apply { addRow(arrayOf("photo.jpg")) })
+            whenever(provider.query(audio, null, null, null, null))
+                .thenReturn(MatrixCursor(arrayOf(OpenableColumns.DISPLAY_NAME)).apply { addRow(arrayOf("song.mp3")) })
+            whenever(provider.getType(image)).thenReturn("image/jpeg")
+            whenever(provider.getType(audio)).thenReturn("audio/mpeg")
+            ShadowContentResolver.registerProviderInternal("files", provider)
+            shadowOf(targetContext.contentResolver).registerInputStreamSupplier(image) { imageData.inputStream() }
+            shadowOf(targetContext.contentResolver).registerInputStreamSupplier(audio) { ByteArray(64) { it.toByte() }.inputStream() }
+            val clip =
+                ClipData(ClipDescription("files", arrayOf("image/jpeg", "audio/mpeg")), ClipData.Item(image)).apply {
+                    addItem(ClipData.Item(audio))
+                }
+            val payload = ContentInfoCompat.Builder(clip, ContentInfoCompat.SOURCE_DRAG_AND_DROP).build()
+
+            val field = editor.editFieldAt(1)!!
+            ViewCompat.performReceiveContent(field, payload)
+            advanceRobolectricLooper()
+
+            val text = field.text.toString()
+            assertContains(text, "<img src=\"photo", message = "the photo is inserted as an image")
+            assertContains(text, "[sound:song", message = "the audio is inserted as sound, not as an image")
+        }
+}

Originally posted by @criticalAY in https://github.com/ankidroid/Anki-Android/pull/23078#pullrequestreview-5944085365

Dominant language
Kotlin
Stars
12.3k
Forks
3.1k
Avg merge
1d 11h
Merged PRs (30d)
383

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ankidroid/Anki-Android

All issues in ankidroid/Anki-Android

Similar issues

More Kotlin issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.