The beginning of a data journey

When I first stepped into the realm of data science and analysis, I didn’t quite know what a dashboard meant. When I started to discover what a dashboard really was, I realized that a dashboard can turn numbers in a table into interesting visuals, like converting numbers to art. Basically, it displays statistical graphs and trends on a computer screen. Then, adding programming to shape those dashboards – that’s where Shiny for Python entered the picture.

Image taken from https://giphy.com/

My first experience with Shiny

I first stumbled upon Shiny for Python just a week before my Data Analyst exam in 2023, plunged into learning it to prepare for my exam. As I began exploring, I discovered that this tool lets you craft interactive data visualizations and dashboards right in Python. Back then, Shiny for Python was a fresh release, launched in 2022, with few tutorials or references to learn from. Moving forward, I turned to YouTube and attempted to recreate Tableau tutorials using shiny instead. At the same time, I found a captivating YouTube video by Matt Dancho, whose video offered a brilliant basic-to-intermediate tutorial on building a financial stock analyzer with Python Shiny. That video remains one of the most informative I’ve seen to this day. Beyond the tutorial itself, he also shared his career journey, revealing how shifting to Python Shiny had skyrocketed his salary from $75K to an impressive $350K per year.

Shiny Assistant – a huge helper

Back in 2023, Shiny Assistant, the AI tool for developing Shiny applications, wasn’t around. I tried ChatGPT,  but its buggy Python Shiny code snippets left me tangled with endless error-fixing struggles. It wasn’t until late 2024, when my teammate, Farhan Bhaiya, introduced Shiny Assistant to me, that things shifted. This AI Assistant became a game-changer for our Python Shiny projects, though it is far from perfect and needs plenty of improvement as it shows some components that are not even discovered in Shiny for Python.

Learning Shiny and the challenges

For a developer, wrapping your head around Shiny for Python – often nicknamed PyShiny – can feel like climbing a steep hill, only to tumble down and climb again. Yet, once you dive into this web framework hands-on, the reward is worth it: you can create stunning, interactive dashboards that data enthusiasts can’t get enough of. That said, the road to mastery is paved with lots of errors and perfecting every component can feel like a battle.

My journey with PyShiny was packed with challenges. Firstly, it uses decorators for each function, and if any decorator isn’t spot on, the dashboard crashes with an error. For non-techies, picture decorators as text-stylists. As an example, take a text – “DaSHbOarD”, add a @lowercase_alphabet  decorator and you get “dashboard”; swap it for the decorator @uppercase_alphabet, and it’s “DASHBOARD”. 

Next, handling tables was a pain, especially with empty values. PyShiny demanded extra code to process data and clean things up, which was needless when processing in a jupyter notebook. Shiny Assistant eventually stepped in and helped tidy the data and fix errors after I supervised the AI Assistant multiple times.

Then, Shiny for Python being still relatively new does not have as many features as R-Shiny which has been around for a decade. Take user authentication, for example: R-Shiny has libraries for login-logout pages, but PyShiny? Nothing built-in. A drag-and-drop option is also available for R shiny dashboards, but PyShiny dashboards are still waiting for that feature. The documentation is very unclear and I had to integrate third-party tools such as Starlette for building a user-authenticated application. Thus, Shiny for Python lacks the ecosystem of related packages showing the limitations of developing web applications. 

Even after creating a user-authenticated application, passing the user credentials from one page to another to track dashboard activity, like a real-life application was tricky to do. Not to mention custom features: I imagined a search engine-like search bar, but with nothing built in Shiny, I had to extensively modify the existing tools, only to fall short of the full functionality I wanted.

Finally, I longed for a reusable button or image template, but rendering the same button or image multiple times turned into a nightmare. Each image or button object demanded a unique ID, so I had to dynamically generate the button IDs and use a different approach to reuse an image.

Why Shiny shines over other data-visualization tools

Dashboards can also be built in Excel, Tableau, PowerBI, and Python competitors Streamlit and R Shiny came first before Python Shiny. Then why choose Python Shiny? Python’s broad range of libraries for data visualization and analysis as well as machine learning, can be integrated in Shiny application which is a good choice for python developers who are not very familiar with the R programming language. Besides, PyShiny can be customized with its web development functionalities whereas the customization of Streamlit is very limited. 

From the experience of one Tableau developer, it was mentioned that the limitations of Tableau are that it can only read data but cannot write data, whereas Shiny can do both. And a third-party interface like Google Sheets was used to give input data in Tableau, but forms can be created in Shiny along with rules in the field. Though the blog mentioned R Shiny, the features are also available in PyShiny.

One of the limitations of Power BI is that it imposes a data size limit. But PyShiny supports big data analysis libraries like polars, thus big data can be loaded and processed in PyShiny. Moreover, Power BI relies on embedding a link in the website where the dashboard made in Power BI is used. Each time an update in the data occurred, it meant rerunning the dashboard in Power BI and updating the embedded link in the website with the new changes took a lot of time. But PyShiny provides real-time access to the data and thus is better than Power BI in many ways.

So, Tableau and PowerBI are good for small-scale applications for their ease and graphical interface where coding is not needed, but for large-scale web applications for maintaining in the long run, a web framework like Shiny is a better choice.

Powerful data visualization and analysis can also be done in Excel, but it has its limitations, such as it is not scalable, maintainable, or secure, and Shiny, being a robust web application, overcomes the error proneness of Excel. Using HTML, CSS, and JavaScript, shiny for Python can customize the look and feel of the app, making it visually appealing and engaging like other web applications.

Final Thoughts

Shiny for Python can be the next big thing for data scientists and analysts. It’s super customizable, great for analyzing data, and can handle big projects, making it stand out from other data analysis tools. After trying it out, I think PyShiny is the combination of web building and data work. To get started with PyShiny, someone does not need a lot of web skills, just a good knowledge of Python is required, and anyone is good to go.

Acknowledgements

Alhamdulillah, Praise be to God. This is my first blog since I started my first job at ARCED Foundation. I would like to thank Pritha Apu, who always inspired me to write this blog, pushing me since 2023 to express my ideas into writing. I’m grateful to Farhan bhaiya and Mehrab bhaiya who have helped me in my learning journey of PyShiny, and to Adnan, who always motivated me when I was stuck with errors. I also used AI to refine and rephrase some parts of this blog for smoother flow of reading.

Website |  + posts