Rapidly create the tools you need.
Scriptflow.io is a dataflow programming system that aims to be the fastest way to create internal tools and visualizations. Scriptflow.io intends to allow users to create flows within minutes and integrate them into their own systems easily. Rather than have a team of developers make one off integrations, data visualizations, or forms for internal use, with scriptflow you can prototype and deploy those systems within minutes rather than days or even weeks. By making it easier to create tools, perform analysis, and visualize data, you can make business decisions faster and save your business time and money while focusing on your customer facing product.
Note: You can run most of the examples on web, but some examples will require the Command Line Interface (CLI) Proxy to be connected. Make sure to install the proxy for the advanced examples that require connection to a database.
Creating a new graph is as simple as clicking on the Create button in the navigation bar of scriptflow.io's homepage. You can create a graph while logged in as well as try scriptflow out anonymously. Note that creating graphs anonymously will make it difficult to find the graph later unless you save the link manually. Creating an account will save all your graphs in your profile and will let you share with team members.
Embedding and Sharing graphs
Graphs as well as dashboards can be embedded and shared with others. This allows you to quickly create a graph, then share it on other websites or with your team.
Read about the editor interface.
Advanced Getting started
Advanced Getting started
Directed Acyclic Graphs
Scriptflow is based on the concept of Directed Acyclic Graphs (DAG). What this means is that the graph always "flows" in one direction (in scriptflow's case - from input to output). This makes it much easier to understand what is going on in a graph as well as view other graphs and understand what they are doing. Scriptflow does not allow for loops in its graphs - this is why it is extremely easy to reason about the flow of your data through a scriptflow graph.
Setting up the Command Line Proxy
Download the appropriate CLI for your system and run by using the following command:
By default the cli runs its proxy on port 12321, you can change this to a different port by using the '-p' option.
scriptflow-cli -p 9081 -w
The proxy uses websockets in order to communicate with clients. This means that if you want to use a load balancer (like Nginx, HAProxy, or Apache) in front of your proxy, you will need to properly configure it to allow websockets (and also allow sticky sessions when load balancing).
For most common usecases, you should setup a single proxy for your team on an internal server. For development, or for individual usecases, you can setup the proxy locally on your own machine.
Read this for more information on system administration with the scriptflow.io proxy.
The graph editor contains a number of elements that can be intimidating for a first time user, but should be simple to understand after a few uses.
The main portion of the graph editor is the graph itself:
Above the graph is the menubar:
To the right of the graph is the node explorer:
In the bottom right corner is the info panel:
If you click on the 'D' button next to the Proxy connector, you can see the dashboard:
You can read more about each element in the graph editor below.
The node explorer has three tabs, "Nodes", "Graph", and "Packages".
The nodes tab shows the available nodes that you can drag and drop into the graph. You can search the available nodes by using the search bar at the top of the Node tab.
Graph Explorer (Tab)
The Graph tab shows the nodes in your current graph. You can drag and drop nodes from this tab into the dashboard when the dashboard is in edit mode. This is important to remember when creating a dashboard since you may be in preview mode.
Package Explorer (Tab)
The Package tab shows packages that you can add into your graph. By default, scriptflow will provide you with the basenodes package to get started, but as more packages are developed, they can add more functionality to scriptflow. For example, the randomdata package adds a node called "RandomData" that will provide a way to easily create fake data to test with.
The info panel provides information on a node or on the graph itself depending on what is selected in the graph. It also provides a Readme panel that can provide a node's entire readme in Markdown format. If you don't know how to use a node, its always useful to read the node's Readme first. If it's not helpful, please tell us in the Forums so that we can improve it.
After clicking on the round 'D' button, you will be shown the dashboard for the graph. You can also view the dashboard by using the menubar's View > Dashboard button. The dashboard is useful for consumers of your graph to quickly use the tool or visualization you've created without seeing the details of how you created it (the graph itself).
Adding a node
Adding a node is as simple as dragging and dropping it from the node explorer. You can also add a node by using the menu bar and choosing Edit > Add Node, then searching for the node you want.
Selecting a node
Simply click on a node to select it. You can also use box select to select multiple nodes at once.
Moving a node
After selecting a node, you can drag a node by clicking and holding on the node, then dragging the node.
You can zoom the editor by scrolling on your trackpad or mouse. In addition, you can use the zoom slider in the top right corner of the editor to zoom as well.
Panning the Editor
You can pan the editor by clicking and dragging on the background of the graph (not on a node).
You can box select by going into Box Select mode (Edit > Box Select), then dragging your mouse across the editor to select multiple nodes. This is useful for moving or deleting multiple nodes at once.
You can select all nodes by using Edit > Select All.
You can delete selected nodes (a single node or multiple) by using your keyboard's Backspace key or using Edit > Delete from the menu bar.
Making a connection between nodes
To make a connection between two nodes, you want to click on a node's input or output circle, then drag and drop the connection onto another node.
Since this is such a common task, scriptflow makes it easier by allowing you to use your keyboard to connect to another node when dragging. After you click on a node's input or output circle, a group of numbers will appear over the other circles that you can connect to. Instead of having to manually drag your connection, you can simply hit the appropriate number on your keyboard to quickly make the connection.
Deleting a connection
To delete a connection, you can either right click on a node's input circle or click on the connection itself to select it, then use the Backspace button to delete the connection.
Note that if you right click on a node's output circle, it will delete all connections attached to that output. Since node inputs can only have one connection attached to them, it is safer to right click on a node's input connection instead.
Cut / Copy / Paste
When selecting a node or a group of nodes, you can use the menubar's Edit > Cut/Copy/Paste commands to perform those common use cases.
When you're done creating a graph and its associated dashboard, you can save your graph by using File > Save. Scriptflow will require that you name your graph and if you have a paid plan, you can determine whether you want to save a private graph or a public graph. For private graphs, you have the option to save credentials directly in the graph - public graphs will never save your credentials.
Learn about organizing graphs.
Saving graphs with tags helps to organize graphs for later.
Tags can be searched when Exploring other graphs and tags also help organize your own graphs.
You can favorite graphs by clicking on the Star icon. Favorite graphs appear in your profile's favorites section.
When you're logged in, you can fork other public graphs by clicking on the 'Fork' link. This will open up the editor with the graph in it allowing you to modify and edit someone else's graph for your own use case. After performing your edits, you can save a new graph using the name, description, and tags that you choose.
View some examples.
Private graphs are only available to accounts that have purchased a developer, team, or enterprise plan.
Saving a private graph
When saving a graph, you will be able to determine your sharing preference - whether you want to make a public graph or a private one. Note that once you have a paid plan, by default your sharing preference is set to private.
Note that users with a developer plan can make private graphs that are hidden from all other users. Users with a team plan can make private graphs from the public, but still share their private graphs with their team. Developer plan users cannot share their private graphs with other users.
Team sharing is only available to accounts that have purchased a team or enterprise plan.
Sharing private graphs with your team
When saving a graph, you will be able to determine sharing preferences. You will also be able to determine whether you want to share only with your team or make a public graph.
Enterprise features are only available to accounts that have purchased an enterprise plan.
Enterprise plans are primarily useful for hosting your own instance of scriptflow on internal servers. If you want the benefits of team sharing without having to host your own server, then it is recommended that you purchase a team plan instead. You can always purchase multiple team plans and organize your users appropriately per team, however, team plans do not support LDAP authentication and are hosted on the same servers as the rest of scriptflow.io.
Enterprise plans support LDAP authentication. Contact us for more information.
Enterprise plans allow you to write private packages that are not shared with public scriptflow.io users. This is accomplished by publishing your package to your self-hosted instance of scriptflow. Since this instance should be hosted on an internal network, this allows you to create private packages that can be shared with your team members, but not with public scriptflow.io users.
This feature is currently only available for enterprise plans.
Contact us for customized enterprise options.
Security and System Administration
Security and System Administration
Security and Credentials
Scriptflow.io takes the security of your data and credentials seriously. Scriptflow will never save credentials for public graphs and the default option is to not save credentials in private graphs. Users have the option to download a json file containing a flow's credentials prior to saving the graph and have an option to load those credentials later. Scriptflow only stores credentials in a graph if the user explicitly requests it - and only for private graphs. Users can choose to save their credentials locally or with another credential management system that works with files. If users want to share credentials, it is recommended to use a secure network storage system to share files.
Credentials are defined in nodes by developers and work seamlessly for graph creators.
Cross-site scripting is a form of attack on a website where a malicious user attempts to retrieve credentials, authorization tokens, or data from unsuspecting users by writing a script that executes on a different domain. Scriptflow.io does not run graphs in the same domain as scriptflow.io. Due to the Same Origin Policy, this means that graphs do not have access to your scriptflow.io credentials or cookies. This is similar to how JSFiddle or other online code tools run.
Note that while scriptflow.io takes precautions to prevent these forms of attacks, users need to be careful to not blindly trust any scriptflow.io graph and should not input their credentials for other websites into scriptflow graphs that they have not created personally or by a member of their team.
System Administration with Scriptflow.io CLI and Proxy
At times it is useful to have a single data proxy that your entire team can use to access server side data. This can be easily achieved by running the scriptflow CLI in proxy mode:
You can specify host and port options by checking the cli help command:
You can then tell your graph creators to setup their graph proxy to the url of the CLI proxy. There are a few things to note when doing this:
- Your proxy server should not be open to the internet. If the user is on your internal network and uses an internal IP to connect to the proxy, the proxy should work for them. If the user is not in your internal network, then the proxy should not work for them. It is not recommended to host a proxy server open to the internet since this could be a security risk for all systems that the proxy server can access.
- If you do not have an internal network, each user can still use their own local proxy. This is the recommended alternative as local proxies are not accessible by other users.
- The proxy uses websockets in order to communicate with clients. This means that if you use a load balancer (like Nginx, HAProxy, or Apache) in front of your proxy, you will need to properly configure it to allow websockets (and also allow sticky sessions when load balancing).
Hosting your own instance of scriptflow.io allows you to keep all your flows on your own network and create private packages that only your employees can access. Private hosting is only available for enterprise plan customers.
Basic Apple Stock Example
The above example shows an embedded graph that displays Apple stock data loaded from a CSV.
Here's a (silent) video showing how this graph was created:
Retrieving weather data from YQL
The above example retrieves weather data from YQL (Yahoo Query Language) and displays it.
View how to connect to common data sources with some data examples.
You can use a Spreadsheet node to manually input data into a table. However, note that scriptflow.io is not optimized for storing data in these spreadsheets (they are mainly for testing manually - use Microsoft Excel or Google Sheets if you need a spreadsheet). You should store your data in an external data source and load the data using a connector instead. The Polar and Radar charts example has an example usage of a spreadsheet.
The example above loads a CSV from a url and displays its results in a table. It then extracts specific columns from the CSV in order to display them in a line chart visualization.
You can load SQLite data in your browser by first loading the scriptflow-sqlite package, then adding the SQLite node into your graph.
HTTP data (JSON, XML)
Load HTTP data using a HTTP or server/HTTPRequest node (depending on if you want to make a request in your browser or via the proxy). Note that making a browser request requires CORS permissions.
The Yahoo YQL example is a good example of making a client side HTTP request and extracting data from it.
Connect to Postgres, MySQL, MariaDB, MSSQL
The above example shows a SQL node connected to a table. A few things to note about this - SQL nodes require a proxy server to be connected in order to function properly. Once a proxy server is connected, you can run parameterized queries against the database and the graph will function like any other scriptflow graph.
Also note the beacon on the node and in the top right corner that indicate that the graph must have the proxy server running. This beacon appears in dashboards as well, so your graph consumers should know to start the proxy for the graph to work properly.
These integrations will be coming soon:
- Amazon Redshift
- Google Cloud SQL
- Elastic Search
- Apache Cassandra
More data sources
Some integrations are still in development, check the Forums for more information. In addition, if you would like to recommend specific integrations, tell us in the Forums or via a support request. Here are some recommendations from others that we are actively working on:
See how to visualize data with scriptflow.
Examples of visualizing data in scriptflow
Add image data easily by using an Image node with a Url source.
Video, Audio, and Iframes
You can use an Iframe or Player node in order to embed videos into your graph and display them in a dashboard. This is useful if you want to have a video play for informational purposes with your tool.
See the Yahoo Weather example for an example of using maps.
User interface examples
User interface examples
Tables and Search UI
Tables are critical when trying to understand the structure of your relational data. Scriptflow tables can be used to view thousands of rows efficiently and can be connected to data sources easily. To add a quick search capability, you can use the TableSearch node to "pipe" and filter your data prior connecting to a Table node.
Pagination, Cards, and Common UI Elements
The above example shows how you can filter data using a Pagination node and the Slice node prior to displaying it in a Table. This is useful for limiting the amount of data displayed to your consumers. The above example also shows how to use a Card node to view a single row of data elegantly.
Browse examples on scriptflow.
What is dataflow programming?
Dataflow programming is a programming paradigm that models a program as a directed graph of data flowing between operations. - Wikipedia
Scriptflow also focuses on unidirectional data flow - this means that your data always flows in one direction - from input to output. This makes it much easier to reason about the state of your flow and makes it so that even non-programmers can create flows. Scriptflow also makes it easy to handle events and asynchronous operations that normally would add complexity to a program.
When to use scriptflow
What is scriptflow really good at?
Scriptflow is extremely good at creating customizable prototypes rapidly. Scriptflow also works well for common data manipulation and visualization tasks - far better than writing custom ad-hoc code or using a domain specific language that can't be exported to the web. It is better than a spreadsheet editor because it natively supports web integrations like connecting to databases or requesting data from HTTP requests without having to write custom script.
With specialized nodes for text, audio, and image data, scriptflow also works extremely well for testing out features using data in those formats or quickly prototyping and embedding a feature into your own website to get feedback from users faster.
What should you use a more specialized tool for?
Always use the best tool for the job.
Scriptflow - and dataflow overall - is not going to remove the need for programmers or work well for creating large monolithic applications. Scriptflow is best suited for small "applets", internal tools, prototyping, and independent features.
It works especially well for non-programmers who want to get started with automation and even works well for experienced developers who want to make quick prototypes. Programmers and non-programmers will discover that they can easily embed and deploy their flows to production or an internal site for rapid usage and user feedback. Programmers don't have to go through code review or do unit testing at the node level because nodes are tested when written.
It can also help in visualizing the logic and data flow through your feature or application.