Welcome to the release of Tapicker v5.0.0. There are many updates in this version that we hope you'll like, some of the key highlights include:
- Introduced a new recipe editor
- Introduced a new execution engine
- Supported custom arguments
- Supported variable definition and reference
- Supported conditions and branches
- Supported random waiting times
- Supported random loops and reverse loops
- Supported extracting data from iframes
- Added more data transformers
- More efficient deduplication algorithm
- And more...
Introduced a new recipe editor
One of the most important updates of Tapicker v5.0.0 is that we have designed a new recipe editor that allows you to create recipes by dragging and dropping, as easy as building blocks.
It provides many useful blocks that you can freely combine to build a powerful workflow with high customization and flexibility.
Introduced a new execution engine
Another equally important thing is that we refactored the core engine and now supports start, pause, resume and stop operations, which is very useful for some time-consuming tasks.
For example, if you start a workflow and have to leave it for some reason, you can safely pause it or stop it directly.
Supported custom arguments
We now support customizing parameters that require user input, which improves the applicability of recipes. This is a super useful feature that allows recipes to be created once and used in multiple places.
As shown below, you can create multiple parameters and then consume them where needed.
In the v3.x version, we preset some parameters, but they are somewhat limited and not extensible. So we improved it in v5.0.
Suppose you create a recipe that can export twitter followers, and you want to customize the profile link every time you start the recipe.
To do this, you add a parameter (named Entry in this example).
This way you can customize it every time you start it.
Supported variable definition and reference
Similar to custom parameters, you can also define multiple variables and then consume them where needed. The difference is that the data of variables comes from the page, while the parameters come from user input.
You can reference a variable by entering two curly braces {{
Supported conditions and branches
Conditions allow recipes to handle data collection in various complex scenarios.
For example, we need to first determine the number of comments on twitter and then decide how to deal with it.
If there are no comments, return, otherwise, scroll the page and scrape the data.
Supported random waiting times
You can set the wait time before each block is executed, it can be a range or fixed. When it is a range (for example: 1 - 5s), a random value will be generated from this range.
Supported random loops and reverse loops
Through loops, you can perform batch repetitive tasks, which greatly improves efficiency. In some websites, robot access will be prevented, and random access can effectively avoid detection by the target website.
- Random - Randomize the execution order, such as: [3, 5, 2, 4, 1]
- Reverse - Reverse the execution order, such as: [5, 4, 3, 2, 1]
Supported extracting data from iframes
Although this case is rare, once you encounter it there is nothing you can do about it. We can now easily extract data from iframes, no extra action is required on your part. It is no different from normal data extraction and is insensitive.
Added more data transformers
When we scrape data from a website, if the data contains unwanted content, we can process the data through a data transformer. For example: extract email address from text, etc.
More efficient deduplication algorithm
We have improved the efficiency of the deduplication algorithm. In data collection, it is particularly important to effectively prevent data duplication. It greatly saves your time checking data.
Conclusion
Due to limited space, there are still many thoughtful features that have not been introduced, waiting for you to discover. There is no doubt that Tapicker v5.0 is an important milestone. It is the transformation from a Web Scraper to Web Automation.
That's all. We continue to improve and strive to do better 💪.
Keep following us!