Martin et al., 2022 - Google Patents

Scangan360: A generative model of realistic scanpaths for 360 images

Martin et al., 2022

View PDF
Document ID
15006901751013501858
Author
Martin D
Serrano A
Bergman A
Wetzstein G
Masia B
Publication year
Publication venue
IEEE Transactions on Visualization and Computer Graphics

External Links

Snippet

Understanding and modeling the dynamics of human gaze behavior in 360° environments is crucial for creating, improving, and developing emerging virtual reality applications. However, recruiting human observers and acquiring enough data to analyze their behavior …
Continue reading at par.nsf.gov (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62—Methods or arrangements for recognition using electronic means
    • G06K9/6217—Design or setup of recognition systems and techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00221—Acquiring or recognising human faces, facial parts, facial sketches, facial expressions
    • G06K9/00268—Feature extraction; Face representation
    • G06K9/00281—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06K—RECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00—Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62—Methods or arrangements for recognition using electronic means
    • G06K9/6267—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30781—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F17/30784—Information retrieval; Database structures therefor; File system structures therefor of video data using features automatically derived from the video content, e.g. descriptors, fingerprints, signatures, genre
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F19/00—Digital computing or data processing equipment or methods, specially adapted for specific applications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass
    • G06N99/005—Learning machines, i.e. computer in which a programme is changed according to experience gained by the machine itself during a complete run

Similar Documents

Publication Publication Date Title
Martin et al. Scangan360: A generative model of realistic scanpaths for 360 images
Xu et al. Predicting head movement in panoramic video: A deep reinforcement learning approach
Yi et al. Audio-driven talking face video generation with learning-based personalized head pose
US12053301B2 (en) Classifying facial expressions using eye-tracking cameras
Park et al. Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory
JP7147078B2 (en) Video frame information labeling method, apparatus, apparatus and computer program
Qiao et al. Viewport-dependent saliency prediction in 360 video
US10614289B2 (en) Facial tracking with classifiers
Supancic III et al. Tracking as online decision-making: Learning a policy from streaming videos with reinforcement learning
JP7476428B2 (en) Image line of sight correction method, device, electronic device, computer-readable storage medium, and computer program
Sharp et al. Accurate, robust, and flexible real-time hand tracking
Zhu et al. Viewing behavior supported visual saliency predictor for 360 degree videos
Wang et al. Predicting camera viewpoint improves cross-dataset generalization for 3d human pose estimation
Chen et al. A probabilistic approach to online eye gaze tracking without explicit personal calibration
Elhayek et al. Fully automatic multi-person human motion capture for vr applications
Chen et al. 3D face reconstruction and gaze tracking in the HMD for virtual interaction
Bernal-Berdun et al. SST-Sal: A spherical spatio-temporal approach for saliency prediction in 360∘ videos
Corona et al. VLOGGER: Multimodal diffusion for embodied avatar synthesis
Li et al. Real-time gaze estimation using a kinect and a HD webcam
Li et al. Predicting user visual attention in virtual reality with a deep learning model
Shi et al. I understand you: Blind 3d human attention inference from the perspective of third-person
Wu et al. Foveated convolutional neural networks for video summarization
Van Gemeren et al. Hands-on: deformable pose and motion models for spatiotemporal localization of fine-grained dyadic interactions
Ni Application of motion tracking technology in movies, television production and photography using big data
Malladi et al. EG-SNIK: a free viewing egocentric gaze dataset and its applications