<a href="#">
  <div align="center">
    <img src="https://user-images.githubusercontent.com/30947706/79588950-17515780-80ee-11ea-8f66-e26da49fa052.png" width='154'/>
  </div>
</a>

<h1 align="center">Ultimate Facebook Scraper (UFS) </h1>

<p align="center">
  Tooling that <b>automates</b> your social media interactions to collect posts, photos, videos, friends, followers and much more on Facebook.
</p>

<hr>

<h2>📢 Important Announcement: UFS 2.0 available (with support for latest Facebook UI)</h2>
<p>Hi guys! The exclusive version of **UFS 2.0** supports scraping for the latest Facebook UI along with many powerful features, much better organization through CSV (Excel-like) file format, bug fixes, and multiple browser support. You can get exclusive access here: 

#### **[Buy UFS 2.0](https://tinyurl.com/UFS-NEW)** 

Building the new version has been a tough technical challenge and I have spent a considerable amount of time and effort on building it. Therefore, kindly support by purchasing it from the link shared [above.](https://tinyurl.com/UFS-NEW) 

⚠️ **The current state of the repo might not be in working state. So, its recommended that you go for the latest version mentioned above.**

<hr>

You can also share small donations through these links:

- <a href="https://buy.paddle.com/product/638653"><b>Platinum (99.99$)</b> -> 2 hours of development<a/>
- <a href="https://buy.paddle.com/product/638651"><b>Gold (49.99$)</b> -> 1 hour of development</a>
- <a href="https://buy.paddle.com/product/638654"><b>Silver (19.99$)</b> -> 30 minutes of development</a>

_If **considerable support is shown through the above donations then I will release the version for the general public** after some time as I want to keep this software open-sourced. So, now is the time for all those folks who benefited from the tool to support its reach for a wider audience :)_
</p>



<p align="center">
  <a href="https://awesomeopensource.com/project/harismuneer/Ultimate-Facebook-Scraper" target="_blank" rel="noopener noreferrer">
    <img src="https://user-images.githubusercontent.com/30947706/76143280-07c70380-6098-11ea-86b1-79fbce03888c.png" width="250" />
  </a>  
  <br>
  <a href="https://wander-lush.org/most-beautiful-places-in-pakistan/">
    <img src="https://media.gettyimages.com/videos/pakistan-flag-video-id1161759219?s=640x640" target="_blank" width="250">
  </a>
  <br><br>
  <a href="https://www.codacy.com/manual/harismuneer/Ultimate-Facebook-Scraper?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=harismuneer/Ultimate-Facebook-Scraper&amp;utm_campaign=Badge_Grade">
    <img src="https://api.codacy.com/project/badge/Grade/7f41790c3c3a4fd29293777c676c8617" />
  </a>
  <a href="#">
    <img src="https://img.shields.io/badge/Build-Passing-brightgreen.svg?style=flat-square&logo=appveyor" />
  </a>
  <a href="#">
    <img src="https://badges.frapsoft.com/os/v1/open-source.svg?v=103" />
  </a>
  <a href="https://www.github.com/harismuneer/Ultimate-Facebook-Scraper/fork">
    <img src="https://img.shields.io/github/forks/harismuneer/Ultimate-Facebook-Scraper.svg?style=social&label=Fork&maxAge=2592000" />
  </a>
  <a href="#">  
    <img src="https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat&label=Contributions&colorA=red&colorB=black" />
  </a>
</p>

<hr>

<h2 align="center">Featured by Top Security Blogs and OSINT Researchers</h2>
<p align="center">
  <br>
  <a href='https://blackarch.org/social.html'>
    <img src = "https://user-images.githubusercontent.com/30947706/76163778-f8fd5100-616a-11ea-97f6-ce648ccbd74b.png" width="160" alt="BlackArch"/>
  </a>

  <img hspace="10"/>
  <a href='https://hakin9.org/ultimate-facebook-scraper-ufs/'>
    <img src = "https://user-images.githubusercontent.com/30947706/76143228-5a53f000-6097-11ea-8a19-9f2eb771ba4b.png" width="120" alt="Hakin9"/>
  </a>

  <img hspace="10"/>
  <a href='https://kalilinuxtutorials.com/ufs-ultimate-facebook-scraper/'>
    <img src = "https://user-images.githubusercontent.com/30947706/76143239-693aa280-6097-11ea-9c27-cfd365a48e55.png" width="120" alt="Kali Linux Tutorials"/>
  </a>
 
 <img hspace="10"/> 
 <a href='https://securityonline.info/ultimate-facebook-scraper/'>
    <img src = "https://user-images.githubusercontent.com/30947706/76143154-c84be780-6096-11ea-932f-345691310628.png" width="150" alt="Security Online"/>
  </a>

<img hspace="10"/>
  <br><br>

  <a href='https://www.kitploit.com/2019/11/ultimate-facebook-scraper-bot-which.html'>
    <img src = "https://user-images.githubusercontent.com/30947706/76143178-0a752900-6097-11ea-8f15-8c8545bb2ff6.png" width="170" alt="Kitploit"/>
  </a>


  <img hspace="5"/>  
  <a href='https://en.kali.tools/all/?tool=2440'>
    <img src = "https://user-images.githubusercontent.com/30947706/76143168-f9c4b300-6096-11ea-97ce-345cddb82945.png" width="140" alt="Kali Tools"/>
  </a>

  <img hspace="5"/>
  <a href='https://www.journaliststoolbox.org/2020/03/04/twitter_resources/'>
    <img src = "https://user-images.githubusercontent.com/30947706/76143186-19f47200-6097-11ea-91d6-3400fa3c5c32.png" width="140" alt="Journalist ToolBox"/>
  </a>

  <img hspace="5"/>
  <a href='https://onehack.us/t/ultimate-facebook-scraper-a-bot-which-scrapes-almost-everything-about-a-facebook/63803'>
    <img src = "https://user-images.githubusercontent.com/30947706/76143245-76579180-6097-11ea-993d-98dd6db76c91.png" width="140" alt="OneHack"/>
  </a>

</p>

<hr>

<h2 align="center">2nd Spot in Top Trending Python Repositories on GitHub </h2>
<p align="center">
  UFS trended among top Python repositories on GitHub for 3 consecutive weeks 🌟
  <br><br>
  <a href='#'>
    <img src = "https://user-images.githubusercontent.com/30947706/76143469-7bb5db80-6099-11ea-9f37-520437e0cddd.png" width="700" alt="Trending on GitHub"/>
  </a>

</p>
<hr>

## News Updates 🏆
* UFS got included as an official tool in the [BlackArch Linux distribution](https://blackarch.org/social.html)! 
* UFS got listed among the [top 20 hacking tools in 2019](https://www.kitploit.com/2019/12/top-20-most-popular-hacking-tools-in.html)!

## Features 🚀

A bot which scrapes almost everything about a user's Facebook profile including:

- uploaded photos
- tagged photos
- videos
- friends list and their profile photos (including Followers, Following, Work Friends, College Friends etc)
- and all public posts/statuses available on the user's timeline
- **NEW:** Now you can scrape Facebook group posts.

Data is scraped in an organized format to be used for educational/research purposes by researchers. This scraper does not use Facebook's Graph API meaning there are no rate limiting issues.

**This tool is being used by thousands of developers weekly and we are pretty amazed at this response! Thank you!🎉**

For **citing/referencing** this tool for your research, check the 'Citation' section below.

## Note 🤝

This tool uses xpaths of **'divs'** to extract data. Since Facebook updates its site frequently, the 'divs' get changed. Consequently, we have to update the divs accordingly to correctly scrape data.

The developers of this tool have devoted time and effort in developing, and maintaining this tool for a long time. **In order to keep this amazing tool alive, we need support from you geeks.**

The code is intuitive and easy to understand, so you can update the relevant xpaths in the code if you find data is not being scraped from profiles. Facebook has most likely updated their site, so please generate a pull request. Much appreciated!

## Sample

<p align="middle">
  <img src="https://user-images.githubusercontent.com/30947706/79589043-34862600-80ee-11ea-9648-5ff86a63e464.png" width="700"/>
 </p>

## Screenshot

<p align="middle">
  <img src="https://user-images.githubusercontent.com/30947706/79589002-259f7380-80ee-11ea-959a-bddf0e2f6629.png" width="700"/>
 </p>

---

## Usage 🔧

### Installation 💻 

You will need to:

- Install latest version of [Google Chrome](https://www.google.com/chrome/).
- Install [Python 3](https://www.python.org/downloads/)
- Have a Facebook account without 2FA enabled

```bash
git clone https://github.com/harismuneer/Ultimate-Facebook-Scraper.git
cd Ultimate-Facebook-Scraper

# Install Python requirements
pip install -e .
```

The code is multi-platform and is tested on both Windows and Linux.
Chrome driver is automatically downloaded using the chromedriver_manager package.

### How to Run

- Fill your Facebook credentials into [`credentials.yaml`](credentials.yaml)
- Edit the [`input.txt`](input.txt) file and add profile, groups and individual group posts links as you want in the following format with each link on a new line:

Make sure the link only contains the username or id number at the end and not any other stuff. Make sure its in the format mentioned above.

Run the `ultimate-facebook-scraper` command ! 🚀

```python
python scraper/scraper.py
```

> Note: There are two modes to download Friends Profile Pics and the user's Photos: Large Size and Small Size. By default they are set to Small Sized Pics because its really quick while Large Size Mode takes time depending on the number of pictures to download.

You can personalize your scrapping needs using the command line arguments:

```bash
python scraper/scraper.py \
    --uploaded_photos True \
    --friends_photos True \
    --friends_small_size True \
    --photos_small_size True \
    --total_scrolls 2500 \
    --scroll_time 8
```

Note that those are the default values so no need to write them down if you're just testing or are okay with them.


## Chromium

Chromium users can add `--chromium True` to run using the Chromium browser.

```bash
python scraper/scraper.py \
    --uploaded_photos True \
    --photos_small_size True \
    --total_scrolls 2500 \
    --scroll_time 8 \
    --chromium True
```


---

## Citation 📚

<a href="https://zenodo.org/badge/latestdoi/145763277">
  <img src="https://zenodo.org/badge/145763277.svg" />
</a>

If you use this tool for your research, then kindly cite it. Click the above badge for more information regarding the complete citation for this tool and diffferent citation formats like IEEE, APA etc.

---

## Important Message ⚠️

This tool is for research purposes only. Hence, the developers of this tool won't be responsible for any misuse of data collected using this tool. Used by many researchers and open source intelligence (OSINT) analysts.

This tool will not works if your account was set up with 2FA. You must disable it before using.

---

## Authors 👋

You can get in touch with us on our LinkedIn Profiles:

#### Haris Muneer

[![LinkedIn Link](https://img.shields.io/badge/Connect-harismuneer-blue.svg?logo=linkedin&longCache=true&style=social&label=Connect)](https://www.linkedin.com/in/harismuneer)

You can also follow my GitHub Profile to stay updated about my latest projects: [![GitHub Follow](https://img.shields.io/badge/Connect-harismuneer-blue.svg?logo=Github&longCache=true&style=social&label=Follow)](https://github.com/harismuneer)

#### Hassaan Elahi

[![LinkedIn Link](https://img.shields.io/badge/Connect-Hassaan--Elahi-blue.svg?logo=linkedin&longCache=true&style=social&label=Connect)](https://www.linkedin.com/in/hassaan-elahi/)

You can also follow my GitHub Profile to stay updated about my latest projects: [![GitHub Follow](https://img.shields.io/badge/Connect-Hassaan--Elahi-blue.svg?logo=Github&longCache=true&style=social&label=Follow)](https://github.com/Hassaan-Elahi)

If you liked the repo then please support it by giving it a star ⭐!

## For Future 🔮

Shoutout to geeks willing to contribute to this project. Please have a look at the [UFS kanban board](https://github.com/harismuneer/Ultimate-Facebook-Scraper/projects/1) for a list of things to be done. 

There are a lot of features that can be added to this tool like adding support for pages, groups, comments etc! Please contribute :)

## Contributions Welcome ✨

![forthebadge](https://forthebadge.com/images/badges/built-with-love.svg)

If you find any bug in the code or have any improvements in mind then feel free to generate a pull request.

> Note: We use [Black](https://pypi.org/project/black/) to lint Python files. Please use it in order to have a valid pull request 😉

## Issues 🔨

[![GitHub Issues](https://img.shields.io/github/issues/harismuneer/Ultimate-Facebook-Scraper.svg?style=flat&label=Issues&maxAge=2592000)](https://www.github.com/harismuneer/Ultimate-Facebook-Scraper/issues)

If you face any issue, you can create a new issue in the Issues Tab and I will be glad to help you out.

## License 📄

[![MIT](https://img.shields.io/cocoapods/l/AFNetworking.svg?style=style&label=License&maxAge=2592000)](LICENSE)

Copyright (c) 2018-present, harismuneer, Hassaan-Elahi
