10-Step guide to schedule your script using cloud services (for free)Ekapope ViriyakovithyaBlockedUnblockFollowFollowingMar 21credit: https://www.
pexels.
com/photo/bandwidth-close-up-computer-connection-1148820/Why can’t I just run it on my laptop?Well…, you can run it locally and I believe many people probably know task scheduler function if you are Windows users.
We set the trigger time and conditions, the script will be run based on our pre-defined criteria.
However, the drawback on this is that you need to keep your computer ‘power on’ at the scheduled time (if not all the time).
I encountered this problem when I wanted to run my tiny script to retrieve weather forecast every morning.
It is not practical for me to keep my laptop power on, just to run this simple task on daily schedule.
Please consider this as the complimentary guide for Dashboarding with Notebooks: Day 3 / Day 4.
The full credit goes to Dr.
Rachael and Kaggle team.
Thanks for their efforts in the original content, they also published live-streaming recorded here.
This article intends to help someone who is not a hardcore programmer and does not familiar with the file system in Bash and cloud services.
Therefore, I included a lot of screenshots and detail explanations here.
Scheduling Python/R scripts using Kaggle and PythonAnywhere cloud servicesKaggle account, we will use the kernels to host and run our Python script.
PythonAnywhere account, we will use the task scheduling to trigger our script hosted on Kaggle.
What do we need to do?Setup Kaggle account and go to ‘Kernels’ tab, then ‘New Kernel’.
You can select Script or Notebook.
It will redirect to the console, with settings tab.
Here, you can upload data, select language (Python/R /RMarkdown), add custom packages and also set GPU/Internet option for your script here.
You can customize your settings here, in my case, I changed option to “Internet connected”.
I also added some packages, you can select your “Docker” as well.
Save and commit your code then return to the Kernels page, you will see your work here.
In this example, my kernel name is ‘WeatherAlertFB’.
‘WeatherAlertFB’ scriptNow, your script is ready…2.
Go to the ‘Account’ tab of your user profile, scroll down until you find ‘Create API Token’.
This will download of kaggle.
json, a file containing your API credentials.
More detail about Kaggle API here.
Open your kaggle.
json file with notepad.
Add the line "proxy": "http://proxy.
server:3128" to the end of your .
json file.
It should look like this.
{“username”:”[KAGGLE USERNAME]”,”key”:”[API KEY]",”proxy”: “http://proxy.
server:3128"}3.
Register to PythonAnywhere.
In your Dashboard page, ‘New console:’ section, click $ Bash.
You will see new Bash console created.
Run the command pip install kaggle –user to install the Kaggle API.
Upload your kaggle.
json file.
Check that your file has been uploaded by running ls.
You should see kaggle.
json listed.
You can also inspect your json file by typing cat kaggle.
json4.
Make .
kaggle directory;mkdir –parents /home/[Your_Python_Anywhere_Username]/.
kagglemove the kaggle.
json file by running the command;mv kaggle.
json /home/[Your_Python_Anywhere_Username]/.
kaggle/kaggle.
jsonkaggle -h will show the help commands.
5.
Make your credentials private by running; chmod 600 /home/[Your_Python_Anywhere_Username]/.
kaggle/kaggle.
json6.
You can search for your kernels, ranked by how recently you ran them, by running; kaggle kernels list –user [YOUR KAGGLE USERNAME HERE] –sort-by dateRunkaggle kernels list — user ekapope — sort-by dateRunPull a copy of your kernel by running; kaggle kernels pull [AUTHOR'S KAGGLE USERNAME]/[KERNEL SLUG FROM URL] -mRunning ls.
You should see that the notebook file and metadata file are both in your current working directory.
kaggle kernels pull ekapope/world-bank-eda-for-dashboard -m7.
Push your notebook by running kaggle kernels pushThis command will re-run the script and commit new version on Kaggle.
It shows that ‘Kernel version 8’ successfully pushed.
Check on Kaggle log, pushed successful.
No errors.
8.
Create new text file using any text editor, put below code in it.
#!/usr/bin/env bashkaggle kernels pull pull [KAGGLE USERNAME]/[KERNEL SLUG] -mkaggle kernels push -p /home/[Your_Python_Anywhere_Username]Save as ‘run_kernel.
sh’, then upload to PythonAnywhere.
run_kernel.
sh fileChecking your files in the root directory.
You should have 3 new files by now.
9.
Run the command chmod u+x run_kernel.
sh.
Then run your script by running .
/run_kernel.
sh.
This should update your kernel on Kaggle.
10.
Go the Task tab on PythonAnywhere to schedule your script.
Times are UTC, not your local time zone.
Enter the file path to your .
sh script.
/home/[Your_Python_Anywhere_Username]/run_kernel.
sh, hit “Create” button.
And…it’s done!.Your script will run daily at the specified UTC time.
Please note that for the free tier, your script will stop running after a month unless you manually renew it, so be careful about that.
Thank you for reading.
Please give it a try, have fun and let me know your feedback!.Get in touch with me to discuss your inspiring projects.
If you like what I did, consider following me on GitHub, Medium, and Twitter.
:P.