This is the first week of my exchange study in Switzerland. The courses here are not as easy as I originally  imagined. Now it's 9.30pm in Swiss local time. I keep sitting in the learning center, browsing the homework assignment this week, planing how to spend my weekend.

I take an Internet course here. Through this course I learn how to connect to the remote cluster, and how to configure jupyter notebook on the cluster to be able to use it remotely.

following  by Chinese :)

首先在linux系統中有專門的commond連線網站伺服器,那就是ssh命令,加密協議實現遠端登入(但是看資料好像並不安全,比如中間人攻擊?)

一般是 ssh -l username website

連線了遠端cluster之後,如果想傳遞檔案,當然就是scp命令(secure copy的縮寫?)

scp [引數] [原路徑] [目標路徑]

不必多表


然後若cluster上安裝有Anaconda(包含python,jupyter notebook等各種包及依賴關係),那麼理論上說就可以遠端的啟動一個jupyter notebook server, 然後在本地就可以通過網站訪問遠端的jupyter notebook. 這樣便省去了自己在本地配置各種工具包各種依賴關係?(或許吧)


反正需要配置一下環境,在cluster設定一下環境變數,然後新建立一個jupyter的配置檔案(設定使用者名稱密碼啥的,之後就可以起一個jupyter notebook server,然後本地通過網站訪問)


之後每一次都只需要先連線cluster,然後在cluster上run一個pyspark,它自動就啟動了一個jupyter notebook server 之後就可以通過本地網站訪問了


值得一提的是jupyter notebook真的好用,可以直接在block裡執行python,正好最近我也要好好學一下python!


以上一些初級階段的理解,以備期末複習:)