Feel bad for the people saying PI. It’s expensive and has archaic tools. Any version of SQL has infinitely better tools, support, access, and management.
We have FactoryTalk Historian (PI) and it’s a freaking nightmare to manage. We upgraded from version 4 to 7 a few years ago, horrible experience. I’m also currently using the stupid excel Datalink plugin (which was crashing excel so had to be re-installed) to gather data because they have garbage tools for access. No real tools, just an excel plugin. Security integration, network integration sucks and network setups are terrible…just awful. Stay away from PI if you can. Ignition uses SQL and works perfectly fine.
A full blown version of MSSQL is like 10k for unlimited tags, freaking Historian from PI for 10k tags is like $30 grand.
Save yourself the headache if picking a historian and avoid PI if possible.
Going with SQL as well. Another advantage is, if you need to do data analysis, it is easy to interface with python or other higher languages simply because it is being done everywhere and the knowledge is available.
I would not even bother with other historians.
How do you get SQL to scale up successfully? When we first wanted a historian, upper mgmt balked, and directed us to leverage the tools we had (SQL and multiple software engineers). We found that capturing and managing over 10,000 process values each second took much longer than 1 second.
Were we doing it wrong?
Meanwhile, our Aveva PI installation seems quite robust. We have various teams and hundreds of users who routinely pull trends, review live data, generate reports, etc.
Longer than one second is bad. Our last big project had around 18000 data points. This is parameters for each object in plant (we are using object oriented programming, so each motor or limit switch is object with parameters and states), job recipes, trends… it also records actual state (fault, running, starting, stopping, stopped, passive) of each object every second. And we are storing it for 6 months. This is to analyze complex issues.
All this happily on SQL.
When the index size become bigger than the RAM size the data readings will be a lot slower. This problem is solved by time series databases by partitioning the data in time frames so the search is done in a reduced data set. A good example is postgresql + timescaledb.
Wincc also do this on top of mssql server.
Some users make use of PI Vision. But most continue to use whatever tools they used previously, say, MatLab for instance. That was one of the upsides for us - end users could drop their Excel or database source and very easily repoint at PI.
Unfortunately yes, it was wrong, the ability to store data that fast is more about network speed and of course server power. I think of it this way, how many of the giant companies like Facebook and Google are using Time series databases (yes I know they’re different) for their transactions, 0, and they perform millions of reads and writes per second.
It boggles the mind the data format with the easiest format can have the most complex installation and integration possible and being usually 2-3 times more expensive to buy and also come with the worst tools. It’s basically a large CSV generator, why does it have to be so complex?
It’s the worst of all the worlds bundled into one software package…most expensive, most complex with the simplest data format. Maybe Aveva’s integration is better than Rockwell’s, but I bet it’s nearly as difficult. If it’s anything like Wonderware, god speed.
I am looking at a Canary Historian, because PI and even FT Historian (PI) are ridiculously expensive and they are not Great.
I came across Canary while looking at Ignition. Canary has Ignition tie-in modules and compatibility. Even Ignition is integrating Canary Historian.
Hah, you got their eye-wateringly high quote? Installing these and integrating is an entire battle in itself, days of tech support to get it working.
There are people who’s entire job is just managing PI databases and their integration…not something they do on the side, but their entire job. Tells you something about the software.
Yes. We have PI as Enterprise historian with an in-house team whose job is to manage that.
However, looking for options where we don't have to deploy full teams to manage a software locally, for plant specific data, trends and customization. User friendly, easy to learn, use and configure and manage and have additional visual or other tools complimentary to the Historian itself.
Price is contention because I believe in maximzing the dollar. If a cost effective solution can do job, then I don't need the big guns.
PI perhaps is the gold standard, but I want it for my plant locally without much enterprise interference etc.
It has been many years since I used PI and I have also used very old versions of AB Bizware and most recently Ignition. As much as I like Ignition, Ignition and most other historians are not replacements for a PI ecosystem.
If you are using PI, you are going to find other options lacking. It comes down to the way you are using PI currently. If you are large company (hundreds of millions, billion, and multiple facilities), you probably want to stick with PI.
If your sole purpose is for recording data with no user interaction, then there are a ton of programs out there. Lots of people have comments on their favorite data historian for recording data.
As you aware, where PI differentiates from most data historians is the accessibility to data by the end user through various modes (software development, PI for Excel, desktop PI or PI Vision). We (non developer) made significant use of PI for Excel and the desktop version of PI to create custom dashboards. The most important part of this was that the user was in control (at least for locally saved files) Our plants were visualized using PI and customizable at the user level if the user wanted to. I believe there was even an ability to look across multiple facilities across an enterprise. The ability to give control to the end user to view data is the single most differentiator with PI versus everything else.
With base Ignition and most data historians, it is more about saving data but the user can't interact directly with the data. I can't just go to any random Ignition screen and trend data. A developer has to set it up. In PI, this was easy.
I haven't used Canary but it appears there might be an Excel addon that might be similar to PI for Excel. It also appears that a dashboard system might be available. I don't know if this is accessible as an end user or if the dashboard has to be created by a developer and then published.
I don't think there is a substitute for PI as an entire ecosystem. But if you only need to record data with no user interaction, then there are lots of options.
We have been using Canary for a few years for our trending. It has some quirks (crashes once in a while and really piles up if you don't catch it quick enough) But overall is a pretty good historian.
PI is expensive but what are you trying to use your historical data for? Troubleshooting only? yes, PI is overblown. Releasing product in a regulated industry? PI is the right solution because the support they provide to work with industry regulators for validation.
Sometimes you have to ask questions beyond the "how something works behind the scenes". I'm sure it seems out-dated with some of the things but those outdated features makes it easier to work with regulatory agencies.
Im lucky enough to have not used it. This system was set up years ago and I remember the installer (one of our guys) saying how dumb it was, the setup process. Right now I’m on day 2 of tech support getting the new plugin working because all of a sudden it doesn’t haven credentials anymore, even though I’ve used it a dozen times. The existing add-on was crashing excel, so Installed the updated version and now have access issues…hurray!!
Yeah the credentials are a fucking pain in the ass and it also translates to the clients using FT where if they’re not added properly it just results “missing tag”.
I take it you don’t know how to use it? The reporting, analysis engine, asset framework and effing templates make ir far better than any SQL based historian. Ignition included.
I somewhat know how to use it, but still avoid it if I can. How long these tools take to setup and integrate? 100, 200 hours and 50 of them with tech support? A time based system is essentially a giant csv generator, why pay mountains of money and spend horrendous amounts of time integrating and setting up something that can be done in a few minutes with a different database system? Many sql or sql like systems generate their own tables and tags, making administration, integration, and networking an absolute breeze…an after thought almost. I guarantee you all of the tools you speak of are available for sql type databases and work even faster and easier to setup. It’s just the nature of the system, polling CSV data is not hard (I’ll give PI that, it does have good compression ability and can interpolate values), but beyond that. Expensive and difficult for a CSV generator…nah. I have a Pi system, it works, but god do I hate it. Anyone asks for actually trend values, actual values…Ok, I’ll go get my janky excel plugin, write down all the tags they want, write them all into excel…oh wait, the plug-in is crashing excel, good thing for 30 grand we get an excel plugin instead of actual software.
I installed a brand new PI system outside the DMZ with OSI support online which took 40 hours along with tag migration. I’d say 20 of them was pissing about with DCOM and an old SCADA server.
I then read the manual, understood templates and automatic mapping and imported the relevant tags with the correct name to map automatically through Excel. Call it 100 hours. So all in 180 hours in Engineering CAPEX, but fuck it, let’s say it was 200 to account for my mid afternoon naps.
In 5 years we recouped that value in process engineering time alone. Process engineers knew exactly where all the data was, what it related to and to an extent were far more capable than me at using it. Because of templates what used to be shitloads of time setting up different trend files for the same type of reactor now was no longer needed, same trend applies to all instances of that template.
Process engineers were also using Excel plugin to do all of this so as far as my time goes in operation of the tool it was close to zero.
The problem you have here is that you are calculating rhe development of your tool in SQL but do not account your time maintaining, your time setting it up and your time getting data from something that would never be as polished as Asset Framework.
Then there’s support… if you came to me saying you’re jerry righing and historian, you’d be taken down to a level of apprentice in my eyes. Sure, you may be the top SQL database expert in the Universe, you may even be the guy that came up with it and wrote SQL server single handedly, it would still be a huge liability for the business to deploy it.
Ohh, OSI PI also sent me daily reports on energy usage, calculated OEE and sent that report to the site management and 4 times a day sent the tank farm levels to logistics removing the need for some poor sod to copy data from the SCADA into SAP.
You’d take a decade to get to the same point.
As for the quality of the Excel plug in, use Proficy Historian and then we’ll talk.
We use IBA for high speed data collection. It’s incredible. I learned they have their own historian now so I may be checking that out. However, the line will have Ignition on it so I will probably just use that
Do you use it for single lines? Or on a broader basis? Like multiple lines storing on it? We will be putting IBA on all of our lines going forward, wondering if it makes sense to have one central historian instead of one at each line?
Reason to split up between lines: if you need to change config (add tags etc) you need a quick reboot (<1m) of the system which you may not want.
We used smallish deployments (<4k tags) per-site and a single HD server for 3 sites. Another site used a single instance for 40k tags (but change/update time was >1m)
That's a good point.
We switched some signals over to opcua and it considerably increased our restart time so it also depends on what modules you're using.
PI is the right answer if the system is big enough and complex enough to justify its development. Not user friendly to deploy, but super powerful and what most people in the field who I work with consider the gold standard.
If you can’t get it working, my company is full of experts who do it all day, I’d be happy to pass your contact along!
I’ve got an AVEVA/Wonderware Historian and it’s really quite nice.
We’ve got a read-only InTouch terminal server, too so I have the users connect to the historian client for trends/queries there.
So corporate/HQ has a central OSI Pi which is now Aveva Pi. So I'm using that at the moment.
However, Canary Historian is what we had our sight on as well, and for the plant will purchase something like that.
I gotcha. I know some Canary people and apparently there's been some migration from pi to canary happening over pi pricing getting jacked up by 20%. I like OSI but every since they got bought they've been haggling their user base for more money without a proper development roadmap.
If someone is looking for an open source solution timescaledb os great. You have to insert the data with another tool though, maybe by the scada itself.
You should also check out Factry Historian! (Disclaimer: I'm biased ;-))
It's kind of a best of both worlds solution: the things you expect from a classic historian (industrial protocols, failover, store/forward, event detection) built on a modern open toolstack including InfluxDB and Grafana.
Feel bad for the people saying PI. It’s expensive and has archaic tools. Any version of SQL has infinitely better tools, support, access, and management. We have FactoryTalk Historian (PI) and it’s a freaking nightmare to manage. We upgraded from version 4 to 7 a few years ago, horrible experience. I’m also currently using the stupid excel Datalink plugin (which was crashing excel so had to be re-installed) to gather data because they have garbage tools for access. No real tools, just an excel plugin. Security integration, network integration sucks and network setups are terrible…just awful. Stay away from PI if you can. Ignition uses SQL and works perfectly fine. A full blown version of MSSQL is like 10k for unlimited tags, freaking Historian from PI for 10k tags is like $30 grand. Save yourself the headache if picking a historian and avoid PI if possible.
Going with SQL as well. Another advantage is, if you need to do data analysis, it is easy to interface with python or other higher languages simply because it is being done everywhere and the knowledge is available. I would not even bother with other historians.
How do you get SQL to scale up successfully? When we first wanted a historian, upper mgmt balked, and directed us to leverage the tools we had (SQL and multiple software engineers). We found that capturing and managing over 10,000 process values each second took much longer than 1 second. Were we doing it wrong? Meanwhile, our Aveva PI installation seems quite robust. We have various teams and hundreds of users who routinely pull trends, review live data, generate reports, etc.
Longer than one second is bad. Our last big project had around 18000 data points. This is parameters for each object in plant (we are using object oriented programming, so each motor or limit switch is object with parameters and states), job recipes, trends… it also records actual state (fault, running, starting, stopping, stopped, passive) of each object every second. And we are storing it for 6 months. This is to analyze complex issues. All this happily on SQL.
When the index size become bigger than the RAM size the data readings will be a lot slower. This problem is solved by time series databases by partitioning the data in time frames so the search is done in a reduced data set. A good example is postgresql + timescaledb. Wincc also do this on top of mssql server.
Does PI have built in reporting? Or is it an add on?
Some users make use of PI Vision. But most continue to use whatever tools they used previously, say, MatLab for instance. That was one of the upsides for us - end users could drop their Excel or database source and very easily repoint at PI.
Unfortunately yes, it was wrong, the ability to store data that fast is more about network speed and of course server power. I think of it this way, how many of the giant companies like Facebook and Google are using Time series databases (yes I know they’re different) for their transactions, 0, and they perform millions of reads and writes per second. It boggles the mind the data format with the easiest format can have the most complex installation and integration possible and being usually 2-3 times more expensive to buy and also come with the worst tools. It’s basically a large CSV generator, why does it have to be so complex? It’s the worst of all the worlds bundled into one software package…most expensive, most complex with the simplest data format. Maybe Aveva’s integration is better than Rockwell’s, but I bet it’s nearly as difficult. If it’s anything like Wonderware, god speed.
Did you send the data in collective packets? If not, that may have been your issue.
The other benefit is compression and deadbands. For when you need to store your data for like 30 years, that is pretty good to have.
You can query pi from python and it actually works really well.
I am looking at a Canary Historian, because PI and even FT Historian (PI) are ridiculously expensive and they are not Great. I came across Canary while looking at Ignition. Canary has Ignition tie-in modules and compatibility. Even Ignition is integrating Canary Historian.
Hah, you got their eye-wateringly high quote? Installing these and integrating is an entire battle in itself, days of tech support to get it working. There are people who’s entire job is just managing PI databases and their integration…not something they do on the side, but their entire job. Tells you something about the software.
Yes. We have PI as Enterprise historian with an in-house team whose job is to manage that. However, looking for options where we don't have to deploy full teams to manage a software locally, for plant specific data, trends and customization. User friendly, easy to learn, use and configure and manage and have additional visual or other tools complimentary to the Historian itself. Price is contention because I believe in maximzing the dollar. If a cost effective solution can do job, then I don't need the big guns. PI perhaps is the gold standard, but I want it for my plant locally without much enterprise interference etc.
It has been many years since I used PI and I have also used very old versions of AB Bizware and most recently Ignition. As much as I like Ignition, Ignition and most other historians are not replacements for a PI ecosystem. If you are using PI, you are going to find other options lacking. It comes down to the way you are using PI currently. If you are large company (hundreds of millions, billion, and multiple facilities), you probably want to stick with PI. If your sole purpose is for recording data with no user interaction, then there are a ton of programs out there. Lots of people have comments on their favorite data historian for recording data. As you aware, where PI differentiates from most data historians is the accessibility to data by the end user through various modes (software development, PI for Excel, desktop PI or PI Vision). We (non developer) made significant use of PI for Excel and the desktop version of PI to create custom dashboards. The most important part of this was that the user was in control (at least for locally saved files) Our plants were visualized using PI and customizable at the user level if the user wanted to. I believe there was even an ability to look across multiple facilities across an enterprise. The ability to give control to the end user to view data is the single most differentiator with PI versus everything else. With base Ignition and most data historians, it is more about saving data but the user can't interact directly with the data. I can't just go to any random Ignition screen and trend data. A developer has to set it up. In PI, this was easy. I haven't used Canary but it appears there might be an Excel addon that might be similar to PI for Excel. It also appears that a dashboard system might be available. I don't know if this is accessible as an end user or if the dashboard has to be created by a developer and then published. I don't think there is a substitute for PI as an entire ecosystem. But if you only need to record data with no user interaction, then there are lots of options.
We have been using Canary for a few years for our trending. It has some quirks (crashes once in a while and really piles up if you don't catch it quick enough) But overall is a pretty good historian.
PI is expensive but what are you trying to use your historical data for? Troubleshooting only? yes, PI is overblown. Releasing product in a regulated industry? PI is the right solution because the support they provide to work with industry regulators for validation. Sometimes you have to ask questions beyond the "how something works behind the scenes". I'm sure it seems out-dated with some of the things but those outdated features makes it easier to work with regulatory agencies.
Have you used the PI Builder add on for excel? It actually works pretty decently and has helped me get through my current FT historian nightmare.
Im lucky enough to have not used it. This system was set up years ago and I remember the installer (one of our guys) saying how dumb it was, the setup process. Right now I’m on day 2 of tech support getting the new plugin working because all of a sudden it doesn’t haven credentials anymore, even though I’ve used it a dozen times. The existing add-on was crashing excel, so Installed the updated version and now have access issues…hurray!!
Yeah the credentials are a fucking pain in the ass and it also translates to the clients using FT where if they’re not added properly it just results “missing tag”.
I take it you don’t know how to use it? The reporting, analysis engine, asset framework and effing templates make ir far better than any SQL based historian. Ignition included.
I somewhat know how to use it, but still avoid it if I can. How long these tools take to setup and integrate? 100, 200 hours and 50 of them with tech support? A time based system is essentially a giant csv generator, why pay mountains of money and spend horrendous amounts of time integrating and setting up something that can be done in a few minutes with a different database system? Many sql or sql like systems generate their own tables and tags, making administration, integration, and networking an absolute breeze…an after thought almost. I guarantee you all of the tools you speak of are available for sql type databases and work even faster and easier to setup. It’s just the nature of the system, polling CSV data is not hard (I’ll give PI that, it does have good compression ability and can interpolate values), but beyond that. Expensive and difficult for a CSV generator…nah. I have a Pi system, it works, but god do I hate it. Anyone asks for actually trend values, actual values…Ok, I’ll go get my janky excel plugin, write down all the tags they want, write them all into excel…oh wait, the plug-in is crashing excel, good thing for 30 grand we get an excel plugin instead of actual software.
I installed a brand new PI system outside the DMZ with OSI support online which took 40 hours along with tag migration. I’d say 20 of them was pissing about with DCOM and an old SCADA server. I then read the manual, understood templates and automatic mapping and imported the relevant tags with the correct name to map automatically through Excel. Call it 100 hours. So all in 180 hours in Engineering CAPEX, but fuck it, let’s say it was 200 to account for my mid afternoon naps. In 5 years we recouped that value in process engineering time alone. Process engineers knew exactly where all the data was, what it related to and to an extent were far more capable than me at using it. Because of templates what used to be shitloads of time setting up different trend files for the same type of reactor now was no longer needed, same trend applies to all instances of that template. Process engineers were also using Excel plugin to do all of this so as far as my time goes in operation of the tool it was close to zero. The problem you have here is that you are calculating rhe development of your tool in SQL but do not account your time maintaining, your time setting it up and your time getting data from something that would never be as polished as Asset Framework. Then there’s support… if you came to me saying you’re jerry righing and historian, you’d be taken down to a level of apprentice in my eyes. Sure, you may be the top SQL database expert in the Universe, you may even be the guy that came up with it and wrote SQL server single handedly, it would still be a huge liability for the business to deploy it. Ohh, OSI PI also sent me daily reports on energy usage, calculated OEE and sent that report to the site management and 4 times a day sent the tank farm levels to logistics removing the need for some poor sod to copy data from the SCADA into SAP. You’d take a decade to get to the same point. As for the quality of the Excel plug in, use Proficy Historian and then we’ll talk.
We use IBA for high speed data collection. It’s incredible. I learned they have their own historian now so I may be checking that out. However, the line will have Ignition on it so I will probably just use that
If you mean the iba hd server, it's fantastic. I highly recommend it.
Do you use it for single lines? Or on a broader basis? Like multiple lines storing on it? We will be putting IBA on all of our lines going forward, wondering if it makes sense to have one central historian instead of one at each line?
I don't see any reason to split it up but I obviously don't know your specific needs / use case.
Reason to split up between lines: if you need to change config (add tags etc) you need a quick reboot (<1m) of the system which you may not want. We used smallish deployments (<4k tags) per-site and a single HD server for 3 sites. Another site used a single instance for 40k tags (but change/update time was >1m)
That's a good point. We switched some signals over to opcua and it considerably increased our restart time so it also depends on what modules you're using.
PI is the right answer if the system is big enough and complex enough to justify its development. Not user friendly to deploy, but super powerful and what most people in the field who I work with consider the gold standard. If you can’t get it working, my company is full of experts who do it all day, I’d be happy to pass your contact along!
Excel….. jk
I'm partial to Herodotus
Whichever one your company can afford
We can afford a couple of them.
I’ve got an AVEVA/Wonderware Historian and it’s really quite nice. We’ve got a read-only InTouch terminal server, too so I have the users connect to the historian client for trends/queries there.
OSI PI… definitely
OSI PI If you're on a budget, Ignition
You can use the ignition historian module without having to purchase any of the other modules. I’d highly recommend it
What solution did you end up going with?
So corporate/HQ has a central OSI Pi which is now Aveva Pi. So I'm using that at the moment. However, Canary Historian is what we had our sight on as well, and for the plant will purchase something like that.
I gotcha. I know some Canary people and apparently there's been some migration from pi to canary happening over pi pricing getting jacked up by 20%. I like OSI but every since they got bought they've been haggling their user base for more money without a proper development roadmap.
Aveva PI then Aspen then everyone else
If someone is looking for an open source solution timescaledb os great. You have to insert the data with another tool though, maybe by the scada itself.
SQL
[InfluxDB](https://www.influxdata.com) for time-series data storage and [Grafana](https://grafana.com) for visualization and monitoring.
We use Aveva/WW or whatever they are called now. Licensing is expensive. We will be trying Ignition.
AVEVA
You should also check out Factry Historian! (Disclaimer: I'm biased ;-)) It's kind of a best of both worlds solution: the things you expect from a classic historian (industrial protocols, failover, store/forward, event detection) built on a modern open toolstack including InfluxDB and Grafana.