Download stack overflow survey data in linux.

Thanks to Stack overflow to make their survey data public. Its best use case for data analysis. I am doing analysis with pandas in jupyter. However like to share way to get data in linux as its pretty straight forward in windows. Or for the never ending issue: Redirecting output to ‘wget-log’.

Stack overflow data: Stack Overflow Annual Developer Survey

To get in linux with our friend "wget"

$wget -O developer_survey_2020.zip "https://drive.google.com/u/0/uc?id=1dfGerWeWkcyQ9GX9x20rdSGj7WtEpzBB&export=download"

-O could be any output file name.

Get URL that shows up after clicking on download button.

Important:

-- The URL has to be in quotes. Without quotes the whole URL expression will be evaluated. Note there is "&" in URL which will put process in background showing message: Redirecting output to ‘wget-log’.

Comments

Move all files except one or few in linux

Moving all files Except one or few. You may come to situation where you have to move file from current folder to some where else. But you also don't want to move all files. Here is how you can do it. $ mv $(ls | grep -v exceptfile.any) ~/destinationfolder/ for Multiple files: $ mv $(ls | grep -v 'exceptfile.any\|exceptfile2.any') ~/destinationfolder/ There could be case your destination folder is current directory. As you will encounter error: mv: cannot move 'destinationfolder' to a subdirectory of itself, 'destinationfolder/destinationfolder' For this you add the destinationfolder in grep parameter as well: $ mv $(ls | grep -v 'exceptfile.any\|exceptfile2.any\|destinationfolder') ~/destinationfolder/ This will work. But incase re-run this command you like to hit output: mv: missing destination file operand after 'developer_survey_2020' Try 'mv --help' for more information. The above out can be overlooked as its correct since there...

Visual Studio Code remote development SSH error in windows

VISUAL CODE remote development. Configuring remote development extension: Refer: Remote development over SSH The steps are straight forward but for error like: "Could not establish connection to "host alias name": The process tried to write to a nonexistent pipe." "Load key "key.pem": Permission denied ubuntu@hostname: Permission denied (publickey)." VSCODE uses CMD for executing SSH commands. The command it uses are: ssh -T -D 53592 -F "C:\Users\<username>\.ssh\config" "host alias name" bash OR "C:\Program Files\Git\usr\bin\ssh.exe" -T -D 53745 -F "C:\Users\<username>\.ssh\config" "host alias name" bash ( if you set remote.ssh path in extension settings) under the hood it executes: ssh ubuntu@ec2-host.compute.amazonaws.com -i "path_to_key.pem" Surprisingly the SSH connection will work via git-bash and putty. Here th...

Sever information

1)To get server information about original installation Date etc... Type below command in command line for windows systeminfo find /i "install date" looks like below in cmd ----------------------------------------------------------------------------- 2) If you are not able to access other machines in domain, even though they are visible in the net work and trying accessing the same machine from other if you get below message like. %"This server's clock " is not synchronized with the primary domain controller's clock" Do the following log off the machine which is not accessible and login with domain administrator and in command line type NET TIME /DOMAIN:%userdomain% /SET and press enter . Now log off and login with your user name and check. 3) To view or change the network configuration of your local computer or a remote computer. You can manually run Netsh commands or you can create batch files or scripts to automate the process. Not only can you ru...

Amol Vijay Shirke

Search This Blog