Publication

2025

1. ICLR

   

   SONICS: Synthetic Or Not–Identifying Counterfeit Songs

   Md Awsafur Rahman^*, Zaber Ibn Abdul Hakim^*, Najibul Haque Sarker^*, and 2 more authors

   International Conference on Learning Representation, 2025

   Abs arXiv HTML

   The recent surge in AI-generated songs presents exciting possibilities and challenges. While these tools democratize music creation, they also necessitate the ability to distinguish between human-composed and AI-generated songs for safeguarding artistic integrity and content curation. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI-generated but the instrumental music is sourced from real songs. However, this approach is inadequate for contemporary end-to-end AI-generated songs where all components (vocals, lyrics, music, and style) could be AI-generated. Additionally, existing datasets lack lyrics-music diversity, long-duration songs, and open fake songs. To address these gaps, we introduce SONICS, a novel dataset for end-to-end Synthetic Song Detection (SSD), comprising over 97k songs with over 49k synthetic songs from popular platforms like Suno and Udio. Furthermore, we highlight the importance of modeling long-range temporal dependencies in songs for effective authenticity detection, an aspect overlooked in existing methods. To capture these patterns, we propose a novel model, SpecTTTra, that is up to 3 times faster and 6 times more memory efficient compared to popular CNN and Transformer-based models while maintaining competitive performance. Finally, we offer both AI-based and Human evaluation benchmarks, addressing another deficiency in current research.

2024

1. CVPR Workshop

   

   Leveraging Generative Language Models for Weakly Supervised Sentence Component Analysis in Video-Language Joint Learning

   Zaber Ibn Abdul Hakim^*, Najibul Haque Sarker^*, Rahul Pratap Singh, and 3 more authors

   Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024

   Abs arXiv HTML

   A thorough comprehension of textual data is a fundamental element in multi-modal video analysis tasks. However recent works have shown that the current models do not achieve a comprehensive understanding of the textual data during the training for the target downstream tasks. Orthogonal to the previous approaches to this limitation we postulate that understanding the significance of the sentence components according to the target task can potentially enhance the performance of the models. Hence we utilize the knowledge of a pre-trained large language model (LLM) to generate text samples from the original ones targeting specific sentence components. We propose a weakly supervised importance estimation module to compute the relative importance of the components and utilize them to improve different video-language tasks. Through rigorous quantitative analysis our proposed method exhibits significant improvement across several video-language tasks. In particular our approach notably enhances video-text retrieval by a relative improvement of 8.3% in video-to-text and 1.4% in text-to-video retrieval over the baselines in terms of R@ 1. Additionally in video moment retrieval average mAP shows a relative improvement ranging from 2.0% to 13.7% across different baselines.

2. NeurIPS Workshop

   

   ENTER: Event Based Interpretable Reasoning for VideoQA

   Hammad Ayyubi, Junzhang Liu, Zhecan Wang, and 8 more authors

   Spotlight at Neural Information Processing Systems Workshops, 2024
3. ICIP

   

   Exploring Attention Mechanisms in Integration of Multi-Modal Information for Sign Language Recognition and Translation

   Zaber Ibn Abdul Hakim, Rasman Mubtasim Swargo, and Muhammad Abdullah Adnan

   2024 IEEE International Conference on Image Processing (ICIP), 2024

   Abs arXiv HTML

   Understanding intricate and fast-paced movements of body parts is essential for the recognition and translation of sign language. The inclusion of additional information intended to identify and locate the moving body parts has been an interesting research topic recently. However, previous works on using multi-modal information raise concerns such as sub-optimal multi-modal feature merging method, or the model itself being too computationally heavy. In our work, we have addressed such issues and used a plugin module based on cross-attention to properly attend to each modality with another. Moreover, we utilized 2-stage training to remove the dependency of separate feature extractors for additional modalities in an end-to-end approach, which reduces the concern about computational complexity. Besides, our additional cross-attention plugin module is very lightweight which doesn’t add significant computational overhead on top of the original baseline. We have evaluated the performance of our approaches on the RWTH-PHOENIX-2014 dataset for sign language recognition and the RWTH-PHOENIX-2014T dataset for the sign language translation task. Our approach reduced the WER by 0.9 on the recognition task and increased the BLEU- 4 scores by 0.8 on the translation task.

2023

1. ICIP

   

   Artifact: A large-scale dataset with artificial and factual images for generalizable and robust synthetic image detection

   Md Awsafur Rahman^*, Bishmoy Paul^*, Najibul Haque Sarker^*, and 2 more authors

   2023 IEEE International Conference on Image Processing (ICIP), 2023

   Abs arXiv HTML Code

   Synthetic image generation has opened up new opportunities but has also created threats in regard to privacy, authenticity, and security. Detecting fake images is of paramount importance to prevent illegal activities, and previous research has shown that generative models leave unique patterns in their synthetic images that can be exploited to detect them. However, the fundamental problem of generalization remains, as even state-of-the-art detectors encounter difficulty when facing generators never seen during training. To assess the generalizability and robustness of synthetic image detectors in the face of real-world impairments, this paper presents a large-scale dataset named ArtiFact, comprising diverse generators, object categories, and real-world challenges. Moreover, the proposed multi-class classification scheme, combined with a filter stride reduction strategy addresses social platform impairments and effectively detects synthetic images from both seen and unseen generators. The proposed solution outperforms other teams by 8.34% on Test 1, 1.26% on Test 2, and 15.08% on Test 3 in the IEEE VIP CUP at ICIP 2022.

2. arXiv

   

   Syn-Att: Synthetic Speech Attribution via Semi-Supervised Unknown Multi-Class Ensemble of CNNs

   Md Awsafur Rahman^*, Bishmoy Paul^*, Najibul Haque Sarker^*, and 2 more authors

   arXiv Preprint, 2023

   Abs arXiv HTML

   With the huge technological advances introduced by deep learning in audio & speech processing, many novel synthetic speech techniques achieved incredible realistic results. As these methods generate realistic fake human voices, they can be used in malicious acts such as people imitation, fake news, spreading, spoofing, media manipulations, etc. Hence, the ability to detect synthetic or natural speech has become an urgent necessity. Moreover, being able to tell which algorithm has been used to generate a synthetic speech track can be of preeminent importance to track down the culprit. In this paper, a novel strategy is proposed to attribute a synthetic speech track to the generator that is used to synthesize it. The proposed detector transforms the audio into log-mel spectrogram, extracts features using CNN, and classifies it between five known and unknown algorithms, utilizing semi-supervision and ensemble to improve its robustness and generalizability significantly. The proposed detector is validated on two evaluation datasets consisting of a total of 18,000 weakly perturbed (Eval 1) & 10,000 strongly perturbed (Eval 2) synthetic speeches. The proposed method outperforms other top teams in accuracy by 12-13% on Eval 2 and 1-2% on Eval 1, in the IEEE SP Cup challenge at ICASSP 2022.

3. Frontiers

   

   Detecting anomalies from liquid transfer videos in automated laboratory setting

   Najibul Haque Sarker, Zaber Abdul Hakim, Ali Dabouei, and 5 more authors

   Frontiers in Molecular Biosciences, 2023

   Abs HTML

   In this work, we address the problem of detecting anomalies in a certain laboratory automation setting. At first, we collect video images of liquid transfer in automated laboratory experiments. We mimic the real-world challenges of developing an anomaly detection model by considering two points. First, the size of the collected dataset is set to be relatively small compared to large-scale video datasets. Second, the dataset has a class imbalance problem where the majority of the collected videos are from abnormal events. Consequently, the existing learning-based video anomaly detection methods do not perform well. To this end, we develop a practical human-engineered feature extraction method to detect anomalies from the liquid transfer video images. Our simple yet effective method outperforms state-of-the-art anomaly detection methods with a notable margin. In particular, the proposed method provides 19% and 76% average improvement in AUC and Equal Error Rate, respectively. Our method also quantifies the anomalies and provides significant benefits for deployment in the real-world experimental setting.