Research Highlights

A transformer-based method for refining and abstracting sketches at different levels of details

Update as of 3 August 2026



A sketch is a rapid, rough line drawing commonly used to capture the essential features of an object or scene and serves as a preliminary step in design fields such as urban planning, architecture, product development, and character design. After initial sketching, artists often refine or abstract drawings to meet varying presentation needs, including adapting to different display resolutions or achieving visual consistency across multiple components. However, defining and controlling levels of detail in sketches remain challenging due to the absence of clear, objective metrics. Existing approaches fall into two main categories: raster-based methods, which treat detail as style but often produce blurred or limited results, and vector-based methods, which support abstraction through hierarchical structures but lack flexibility for refinement and semantic understanding.

This project proposes a semantic-driven framework to address these limitations. By distinguishing between structural and decorative components of sketches, the method applies different refinement or abstraction strategies while preserving overall meaning. A transformer-based representation encodes sketches into vector form, followed by decomposition using adversarial and contrastive learning techniques. Dedicated operators are then applied within the embedding space to manipulate detail levels effectively. The proposed approach enhances flexibility, consistency, and semantic integrity, offering valuable applications for both industry and academic research, while advancing skills development in artificial intelligence and digital design.
 


Team Members:

  1. PI: Dr. LIU Xueting, Yam Pak Charitable Foundation School of Computing and Information Sciences, Saint Francis University


Reference no.: UGC/FDS11/E02/24