Setting up this website
The following is a timeline of what I did in each session while editing the site itself. This includes any scripts, solutions to issues, or whatever settings I had to set to get this site up and running.
March 18th
While I made an account in the beginning of January, it has remained dormant until this date. I want this site to be an archive and journal for notes, projects I've done, and it'll hopefully become a resume goldmine.
After donating the amount to make me an Arpa member for SDF.org, I am able to make a website. First, I edited the prompt of the terminal since the prompt only presented the following:
$
In addition, the I found the shell to be the korn shell. Since I do not mind the shell I'm going to edit the .profile file to edit the prompt to one that I like. Within the file some predefined variables were given by SDF org so I utilized those to create the following prompt:
[jran@sdf.org] [current directory]:$
I'm too lazy to write out the full current directory but its a long directory since sdf organizes by member, first letter of the username, and then the actual username.
An issue I ran into was the misuse of ' ' and " " when using functions of a shell. I thought it was simliar to programming where they both represent a string but this is wrong as they both have a different effect on the content within.
For example:
"$PWD" # Literal String
'$PWD' # Shell recognizes the environmental variable $PWD
Other than that interesting tip, the next step was actually setting up the website using the following:
$ mkhomepg
$ mkhomepg -a
The first command made the actual website and set a symbolic link to another directory for hosting websites. The second command sets the custom url of which I defined as jran.sdf.org
April 18th
My website runs fine, but I want to utilize a markdown static website generator. However, it seems that the most essential part of installing is not availble for me in sdf.org. Which is to pip install mkdocs. Everytime I use pip it simply denies me access.
As a result, I searched through the boards of sdf and found that in order to get pip working I need to enter:
$ python -m pip install --user somepackage
This command calls on pip directly through python and sets it within a hidden directory in the user's home directory. In order to get it to properly run I added the path to the python files into my PATH variable within the .profile file.
PATH=$HOME/.local/bin:$PATH
However, before we continue, I want to explain the command that I found on the sdf board for various questions. I am not familiar with three specific parts of the command which is the following: python -m option and the --user option.
After some more research, I found that the -m switch is specifically used to call on a module as a script from the commandline by using the module name rather than the filename which may not be obtainable in some cases. In my case, sdf would not allow me to execute pip on its own, so I had to use python -m to call on pip and combine it with the --user option to isolate the packages within my local directory instead of the greater sdf ecosystem.
Onwards to the actual site, I installed the mkdocs package within my home directory and simply followed the installation guide to creating the new project to build the site. I called the directory testHTML and it proceeded to create a single mkdocs.yml file and a directory that'll contain the markdown files that serve as a template for the site.
Essentially what mkdocs does is it intakes markdown files and outputs full static site that is ready to be hosted immediately. This is extremely useful for simple sites such as an archive-orientated site where files can be stored and entire pages created to document those files.
April 20th
Today, I built and deployed the MkDocs website, the process is not as difficult as I thought. I simply used the following:
$ mkdocs build
This generated an entire directory that contained the complete site and I was able to simply move those files to the directory that hosts the website under sdf. Recall that this is within my home directory as its a symbolic link pointing to the actual host location within the sdf system.
When loading the site with jran.sdf.org, it was a horrible mess of html and was stripped of javascript functionality or css styling. However, the most important part was that the site worked and I needed to figure out why the finished build was a mess of html. To solve this I immediately checked the permissions of the files and sure enough they were improperly set like the following.
$ lla .
$ drwx------ 2 jran nobody 512 May 9 13:33 js
$ drwx------ 3 jran nobody 512 May 9 11:47 css
$ -rwx------ 1 jran nobody 9028 May 9 13:33 index.html
The main problem is that the permissions were only given to the owner, which is not desirable for a web server.
$ chmod 754 *
This command sets the permission for every file as files like the index.html needed read permission. Checking back on my website, the mkdocs static site was completely functional.
After some expirementing, I decided to figure out how to filter my directories as they need to have a executable bit but not a readable bit to ensure that no users can browse the directory of the entire site without permission. I used the following command to change all permissions of directories.
$ find ./* -type d -exec chmod 751 {} +
This basically searches the current directory and recognizes all files and directories, the next option filters and returns only directories, the next option adds command for each and every one of the results from find that has been filtered.
However, I was still able to browse directories within the site which can be troublesome. I'll figure it out later.
May 5th
https://github.com/gristlabs/mkdocs-windmill --Possible theme https://sdf.org/?tutorials/htaccess#intro
Using SDF's tutorial section, I was able to find the above link to an introduction to the ".htaccess" file. This taught me how to configure the server using dynamic configuration for the Apache HTTP server. For now, I just want to deny users from using directory listing, so I entered the following directive:
Options -Indexes
Testing the changes, it works, my website is able to operate while denying directory listing to unauthorized users.
Next on my list is getting my website secured. In other words, making the website communicate over HTTPS instead of the default HTTP.
However, after some reading, I found that to secure my website I must upgrade my membership, of which I'll get to at a later time.
Next I started building the script to automatically build my site. However, it seems that I've broken the site somehow when testing how to deploy. File permissions aren't working and the website is once again stripped down. Immediate instinct is looking at the permissions, they are properly set, however, there are some anomalies. First, The group owners are different, one group is nobody while the other is under an ARPA group. Immediately I execute, mkhomepg -p, which in sdf resets the permissions to default and group owners. This effectively gives me a clean slate of permissions for the files and directories. However, that was not the solution as after configuring the proper permissions, the site was still botched.
I instead choose to focus on making a script that automates the deployment of my site. Previously, I've been moving the files and setting the permissions manually. The script simply consists of the following core code:
mkdocs build
mv ~/testHTML/site ~/html
mkhomepg -p
The last line is to reset all permissions and owners of the directories and files within the html directory. I found this an easier method of setting permissions opposed to my arbituary solution using find.
However, it seems this method broke my site as after executing the script my site was stripped of js and css functionality again. I immediately checked the permissions, but found the anomaly this time wasn't the permissions but the different modified dates. Note that is has been about a week since I've done anything with my website. I immediately recognized that the mv command wasn't replacing all the files and kept some unchanged files which could cause trouble since the automation tool merged two completely different sites, one where I only had an index.html file and another where I have a vague template for purpose of an archive site. While I can't pinpoint the exact code that may be causing this issue, its far better to remove all of the files under html and replace it with the new build of the site each time I deploy since that new build is alreadly guaranteed to work assuming I formatted the mkdocs.yml and markdown files correctly. To solve this I simply added the following:
rm -r ~/html/*
This deletes all of the files under the html directory whilst keeping the actual html directory. This should happen before I move the testHTML files into the html directory.
The script is complete and now I can completely focus on building the actual content of the site.
May 7th
Another addition I added was adding my script to the PATH environmental variable. Instead changing directories to where the script is located I decided to create a bin directory to host all future scripts so I can execute them anywhere on the system. This means I can be deep within the testHTML directory and execute the script without changing directories and it'll deploy on its own.
PATH=$HOME/bin:$HOME/.local/bin:$PATH
This is within my .profile file, so now I'm able to execute the deploy script no matter my current directory.
In addition, I've added a few options to the script to make it properly work. The only line I edited was the mkdocs line and it is now
mkdocs -f ~/testHTML/mkdocs.yml -d ~/testHTML/site
The -f option specifies the mkdocs.yml otherwise the script and mkdocs.yml need to be in the same directory. The -d option specifies where to place the finished build of the site, otherwise it'll place the build within the ~/bin directory.