Beyond Scrolls & Screens

Beyond Scrolls & Screens turns machine-generated object labels into stepping stones between Japanese screens and picture scrolls. Choose a motif, hop to its counterpart in another work, then zoom back out for museum context. The labels are fallible, but that imperfection can sharpen rather than replace close looking.

What is Beyond Scrolls & Screens?

Three museum collections supply the raw terrain: Tokyo National Museum, Tokyo Fuji Art Museum, and Tachibana Museum. Their scrolls and folding screens were digitized at high resolution, including a work the project describes as 12 metres long. Machine vision marks details such as trees, bridges, faces, clothing, flowers, and animals; selecting one carries the visitor toward similarly labeled fragments in another picture. The result is less a gallery line than a visual relay race across works separated by centuries. Independent testing also found useful cracks in the system: a supernatural figure could become a foot or an animal, and patterns on clothing could be mistaken for the thing depicted. Those errors are reminders to use the labels as prompts for attention, not as catalog truth.

What you can do there

• Pan and zoom across Japanese screens and scrolls
• Select machine-generated motif labels
• Jump between matching details in different works
• Open artwork and museum context

Why we picked it

The experiment gives a familiar computer-vision trick an unusually graceful purpose. Instead of stopping at object detection, it turns each label into a path between museums and artworks. That creates a reason to inspect brushwork, pattern, and composition closely while also making the model's blind spots visible enough to question.

How to get the most from it

Switch to English or Japanese before starting, then allow the short introduction to finish if Skip is unresponsive. Fullscreen gives the long works more room. Pick one concrete label—tree, bridge, face, or animal—and follow it through several images before opening the museum detail page to check the artwork's actual title, date, maker, and context.

Good to know

The live shell recommends fullscreen and exposes English and Japanese options. Machine labels are navigational hints, not scholarly descriptions, and an older hands-on review documented misclassifications. Keyboard, touch, screen-reader, audio, privacy, and performance behavior were not verified. Some imagery includes a severed head and supernatural figures.

Who made it, and when?

The earliest supported launch date we found is February 2020.