“Web Scraping Technology Is not Only for Businesses but also for Solving Social Problems” Says Juras Juršėnas

For over seven years, Oxylabs has been the go-to supplier of premium proxies and public web data-gathering solutions, helping businesses of all sizes harness the potential of big data. Juras Juršėnas, Chief Operating Officer at Oxylabs, with over 16 years of experience in the field, has established himself as an expert in IT and product management. His ability to apply strategic problem-solving, critical thinking, and people management skills have led him to occupy the position of COO at Oxylabs, a leading global provider of premium proxies and public web data scraping solutions. Juras work routine revolves around innovation management, which often includes doing something that has never been done before. He is passionate about technology and the possibilities that it brings. In an exclusive interview with Analytics Insight, Juras shared his perspective on the company, its achievements, challenges, and the future of the ethical web scraping industry.

1.Kindly brief us about the company, its specialization, and the services that your company offers.

Oxylabs is the leading provider of tools and solutions for large-scale public web data gathering. Providing infrastructure for ethical web scraping is integral to our day-to-day operations.

I joined the company almost four years ago, and to this day, it's fascinating to be a part of the Oxylabs team. In my opinion, our conscious commitment to innovation and ethics allows us to stand out from the rest of the competition. It's rewarding to work with all our business partners, whether Fortune Global 500 companies or startups looking to be the next unicorns. Please brief us about the products/services/solutions you provide to your customers and how they get value. We provide tools and solutions for businesses looking to gather publicly available data on a large scale. Our product catalog includes proxies and ready-to-use web data-gathering solutions, such as Scraper APIs.

Our typical proxy infrastructure clients are large enterprises with in-house resources to run their web scraping activities. They only need our extensive proxy network infrastructure to distribute their data request or get specific geo-located data. Other companies choose ready-to-use tools such as Scraper APIs that are perfect for businesses that prefer to work on analyzing data instead of challenging data acquisition. The solution consists of three distinct products – E-commerce Scraper API, SERP Scraper API, and Web Scraper API – each designed to collect public web data from different sources on the internet. The simplified process is specifically advantageous to smaller companies to reach the results faster and stay competitive on business insights with large corporations.

Web scraping is used extensively in e-commerce. E-commerce businesses gather data for market research, competitive analysis, attempting to understand consumer sentiment, and predicting which goods will be trending.

Financial firms also use web scraping to analyze and evaluate enterprises and find new clients. These companies depend on technology for risk management and due diligence.

Meanwhile, for certain businesses, web scraping is the basis of their operations. For example, travel fare aggregators and pricing comparison websites rely on this technology.

To sum up, we have the most extensive proxy network infrastructure and the most diverse range of IP addresses from various countries and cities. While our ready-to-use solutions effortlessly deliver web data to our clients.

2.What is your biggest USP that differentiates the company from competitors?

As mentioned, we put a lot of effort into research and development. We assembled an AI/ML Advisory Board of five industry and academic leaders, including Stripe and ex-MIT/NASA representatives. The board assists Oxylabs in product development processes, pushing the boundaries of ethical web scraping technology.

We take great pride in the fact that our team is consistently developing new solutions. As a result of our efforts, we hold dozens of patents for our solutions and infrastructure. Web scraping is no easy technology, and often unexpected things happen. Web scrapers break, and parsing pipelines encounter issues due to ever-changing website layouts. Hence, since day one, we consciously emphasized innovation to clear all the roadblocks in the background. Another USP would concern our ethical approach to everything that we do. For example, the procurement of residential proxies. These proxies redirect internet traffic through physical devices owned by real people. To implement a fair practice model, users must give recorded, explicit consent, and network participants must be compensated if possible. Well, this is our stance. Regrettably, many businesses enable methods that make people unaware that their device is an active proxy (exit node) for a third party.

To guarantee that a fair practice model is implemented, we have created a Tier A+ model that marks all fair practice checks: securing explicit consent and fully informing and rewarding users for participating in the proxy network. Mention some of the awards, achievements, recognitions, and clients' feedback that you feel are notable and valuable for the company.

We see an increasing need to show how critical web scraping technology is not only for businesses but for solving social problems. Therefore, we started a new pro bono program called Project 4ß as part of these efforts. Through 4ß, Oxylabs offers free technical know-how, public web data collection infrastructure, and resources on a pro-bono basis to academia and non-profit institutions.

For example, after winning the Govtech Lab Challenge, Oxylabs partnered with RRT – a Lithuanian organization that oversees Lithuania's electronic, postal, and rail sectors. It protects the internet against unlawful and dangerous information. The challenge was to automate unlawful content identification, particularly related to child sexual abuse or pornography, in the Lithuanian IP address space. Oxylabs produced a one-of-a-kind specialized tool that RRT had integrated into their regular operations at the beginning of 2022.

On the business side, Oxylabs was named Proxyway's best proxy provider a few years in a row. Also, we've been recognized as Europe's fastest-growing public web data acquisition provider by Financial Times in 2022. These recognitions wouldn't be possible without our fantastic 400 employees' dedication to unlocking new heights, huge kudos to them.

3.Kindly mention some of the major challenges the company has faced till now.

It usually takes a lot of work to develop web scraping solutions. Being a pioneer is an incredible feeling but makes tasks much more demanding. Being a front-runner entails setting the pace and putting in the work. It implies that many other companies look up to us and emulate the models we develop.

We take satisfaction in assisting clients in maintaining an uninterrupted flow of data. Supporting infrastructure, writing, scraping code, and everything else takes resources. With the increased novelty of the sector, we are constantly confronted with emerging issues.

Also, web scraping has yet to gain traction in the public consciousness. To this day, there is little legislation in place throughout the world.

Hence, to continue safeguarding our industry from within, together with four other organizations, we have established an Ethical Web Data Collection Association. The EWDC represents the interests of companies that rely on web scraping technology.

Our objective is to gather these companies together and advocate for best practices, contribute to developing industry standards, and raise public knowledge about how crucial web scraping technology is to businesses and consumers alike.

4.Where do you see growth coming in for the industry?

While there are established industries such as e-commerce and finance, there are also new ways to employ collected data. Most data came from internal sources in the past, with some external vendors rounding out the picture. Recently, however, there has been a shift toward online scraping as the primary method for automating alternative data collection.

The alternative data industry is worth almost $3 billion. Nevertheless, the industry is still in a nascent stage. In our new research, we noted the trend of an ever-growing demand for public data to gain insights and stay relevant.