Conference Paper2025
Advancing Dark Web Data Extraction: An Extensible Framework for Multimodal Threat Intelligence
Afzal Hossain Alif, Mojtoba Zaman Mantaka, Rubaiyat Islam, Md Rezaul Islam, Saadia Binte Alam
2025 28th International Conference on Computer and Information Technology (ICCIT)
In: 2025 28th International Conference on Computer and Information Technology (ICCIT)
IEEE, pp. 1864-1869
Abstract
Crawling the dark web presents significant challenges due to the complex approaches required for exploring and extracting data from sites hosted in the Tor network. This paper introduces Deep Crawler, a configurable and extensible dark web crawler designed to perform comprehensive data collection and extraction, which integrates with the Tor network for anonymity. Our framework employs specialized extraction modules to systematically identify and structure threat intelligence, including forms, contact information, and potential indicators of illicit activity. We demonstrate how Deep Crawler's advanced multimodal data extraction capabilities, including specialized identity document and financial data extractors, enable the discovery and analysis of threats such as stolen Personal Identification Data (PID) and financial information, offering a valuable intelligence gathering framework for researchers and law enforcement.
Keywords
Deep WebWeb applicationComponent (thermodynamics)The InternetKey (lock)
References
- 1.Luciano Barbosa, Juliana Freire. (2007). An adaptive crawler for locating hidden-Web entry points. , 441–450[10.1145/1242572.1242632]
- 2.Victor Benjamin, Joseph S. Valacich, Hsinchun Chen. (2019). DICE-E: A Framework for Conducting Darknet Identification, Collection, Evaluation with Ethics1. MIS Quarterly, 43(1), 1–22[10.25300/misq/2019/13808]
- 3.Matthias Schafer, Markus Fuchs, Martin Strohmeier, Markus Engel, Marc Liechti, Vincent Lenders. (2019). BlackWidow: Monitoring the Dark Web for Cyber Security Information. , 1–21[10.23919/cycon.2019.8756845]
- 4.Abdul Hadi M. Alaidi, Roa'a M. Al_airaji, Haider Th.Salim Alrikabi, Ibtisam A. Aljazaery, Saif Hameed Abbood. (2022). Dark Web Illegal Activities Crawling and Classifying Using Data Mining Techniques. International Journal of Interactive Mobile Technologies (iJIM), 16(10), 122–139[10.3991/ijim.v16i10.30209]
- 5.Javeriah Saleem, Rafiqul Islam, Muhammad Ashad Kabir. (2022). The Anonymity of the Dark Web: A Survey. IEEE Access, 10, 33628–33660[10.1109/access.2022.3161547]
- 6.Jesper Bergman, Oliver B. Popov. (2023). Exploring Dark Web Crawlers: A Systematic Literature Review of Dark Web Crawlers and Their Implementation. IEEE Access, 11, 35914–35933[10.1109/access.2023.3255165]
- 7.Yizhi Liu, Fang Yu Lin, Zara Ahmad-Post, Mohammadreza Ebrahimi, Ning Zhang, James Lee Hu, Jingyu Xin, Weifeng Li, Hsinchun Chen. (2020). Identifying, Collecting, and Monitoring Personally Identifiable Information: From the Dark Web to the Surface Web. , 1–6[10.1109/isi49825.2020.9280540]
- 8.Ahmed T. Zulkarnine, Richard Frank, Bryan Monk, Julianna Mitchell, Garth Davies. (2016). Surfacing collaborated networks in dark web to find illicit and criminal content. , 109–114[10.1109/isi.2016.7745452]
- 9.Ciara Devlin, Scott Chadwick, Sébastien Moret, Simon Baechler, Quentin Rossy, Marie Morelato. (2024). Illuminating the dark web market of fraudulent identity documents and personal information: An international and Australian perspective. Forensic Science International, 363, 112203[10.1016/j.forsciint.2024.112203]
- 10.Bogdan Covrig, Enrique Barrueco Mikelarena, Constanta Rosca, Catalina Goanta, Gerasimos Spanakis, Apostolis Zarras. (2022). Upside Down: Exploring the Ecosystem of Dark Web Data Markets. IFIP advances in information and communication technology, 489–506[10.1007/978-3-031-06975-8_28]