Linux

By rbosaz, 12 September, 2024

The following was taken from here. All below steps assume a Linux O/S. I used Debian 12.

Setup Ollama:

curl -fsSL https://ollama.com/install.sh | sh

Running Model:

ollama run llama3

To use the ollama API, you'll need to open appropriate ports and perform the following extracted from here:

We want our API endpoint to be reachable by the rest of the LAN. For ollama, this means setting OLLAMA_HOST=0.0.0.0 in the ollama.service.

  • Run the following command to edit the service:

    systemctl edit ollama.service

Find the [Service] section and add Environment="OLLAMA_HOST=0.0.0.0" under it. It should look like this:

[Service] Environment="OLLAMA_HOST=0.0.0.0"

  • Save and exit.
  • Reload the environment.

    systemctl daemon-reload systemctl restart ollama

Example Ollama API is as follows:

curl http://192.168.1.179:11434/api/generate -d '{
 "model": "codellama",
 "prompt": "Why is the sky blue?",
 "stream": false
}'
 

Adding a web UI

One of the easiest ways to add a web UI is to use a project called Open UI. With Open UI, you can add an eerily similar web frontend as used by OpenAI.

You can run the web UI using the OpenUI project inside of Docker. According to the official documentation from Open WebUI, you can use the following command if Ollama is on the same computer:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Tags

By rbosaz, 6 September, 2024

Disables GUI on boot (CLI):

sudo systemctl set-default multi-user

Enables GUI on boot (Graphical):

sudo systemctl set-default graphical

Start GUI from command line:

sudo systemctl start gdm3

Tags

By rbosaz, 28 March, 2024

Byobu is a GPLv3 open source text-based window manager and terminal multiplexer.

byobu

For loop to unzip a bunch of zips files in the same directory.

#!/bin/bash

#Stop shell splitting on spaces
ORIGINAL_IFS=$IFS #internal field separator
IFS=$(echo -n "\n")
for FILENAME in $(ls *.zip)
do
    echo "Unzipping $FILENAME..."
    unzip -o $FILENAME
done
IFS=$ORIGINAL_IFS

Determine size on disk of directories.

for FOLDER in $(ls); do du -sh --apparent-size $FOLDER; done 

The following will sort it by size.

for FOLDER in $(ls); do du -sh –apparent-size $FOLDER; done | sort -h

Determine size of specefic folder.

du -sh --apparent-size <folder_name>

Make list of all files in a directory, recursively.

find . -type f > ../filenames.txt

Given text document containing a listing of directories and filenames (paths). One can use the following commands to search for filename types. This will output a count of files with filenames ending with .docx.

cat ../filenames.txt | grep -i .docx$ | wc -l

Give the same document as the preceding command. One can use the following commands to search for filename types. This will output a count of files with filenames ending with .docx where the path begins with <directory_name. 

cat ../filenames.txt | grep ^./<directory_name>/ | grep -i .docx$ | wc -l

To search CSV files containing "find this,"  use the below command. The -i argument in this command makes the search case insensitive. The */*.csv argument is the path to search tell grep to open every folder, then each file within those folders that ends in .csv, and search for the "find this" keyword.

grep -i "find this" */*.csv

If you have docker and docker-compose installed, start an Ubuntu container as follows.

sudo docker run -it ubuntu:latest bash

Remove all stopped containers at once.

sudo docker container prune

Display all running Docker containers.

sudo docker ps

Kill a running docker container.

sudo docker kill <container_name>

Start an Ubuntu docker container by mount a volume for persistent data. Mount type will be volume.

sudo docker run -it --mount type=volume,src=test-data,dst=/mnt
ubuntu:latest bash

Display docker volumes.

sudo docker volume ls

Remove docker volumes. Note volumes can only be removed if containers have been completely removed.

sudo docker volume rm <volume_name>

Start an Kali docker container by mount a volume for persistent data. Mount type will be bind.

sudo docker run -it --mount type=bind,src=/host/folder,dst=/container/folder
kalilinux/kali-rolling bash

Start a docker container passing database login environment variables to be used by the container. 

sudo docker run -it -e DB_USER=root -e DB_PASSWORD=yourefired ubuntu:latest bash

Verify environmental variables in container.

echo $DB_PASSWORD

Start jupyter docker container with forwarded ports to host. On the host use localhost/127.0.01 with host port to connect to container port.

sudo docker run -p host_port:container_port jupyter/scipy-notebook:latest

Docker can create a lot of files. To free up space quickly run the following command. Since this command doesn’t delete volumes, it won’t delete any of your important data.

sudo docker system prune -a

Services defined in the same docker-compose.yaml file will be part of the same docker network.

To drop into a container's bash, run the following command.

sudo docker-compose run --rm <name_of_container> bash
Running your own local Aleph server with Docker on Linux

Reference Hacks, Leaks, and Revelation Book by Micah Lee

First obtain docker-compose.yml and aleph.env.tmpl files from GitHub. Copy files to the directory you'll use as the Aleph docker root (i.e., aleph). 

Comment out - "~:/host" in docker-compose.yml. 

Rename aleph.env.tmpl to aleph.env. If only one user will be using this server update ALEPH_SINGLE_USER in aleph.env to true. 

Using the below command, update result for ALEPH_SECRET_KEY in aleph.env.

openssl rand -hex 24

Ensure your computers vm.max_map_count is greater or equal to 262144. Check vm.map_count value:

sudo sysctl vm.max_map_count

Update vm.max_map_count value:

sudo sysctl -w vm.max_map_count=262144

Start server.

sudo docker-compose up

If first time starting aleph docker instance then, in a separate window, run following command after above command finishes. Once below command finishes, you should be all set.

sudo docker-compose run --rm shell aleph upgrade

To administer your aleph instance you must use bash terminal. The following command allows you to do that.

sudo docker-compose run --rm shell bash

Run the following to see the list of commands Aleph supports. And if you want to learn more about a command add --help on it.

aleph --help

Adding data to Aleph. First ensure your Aleph instance is running, then run the following command. This command will provide readonly (ro) access to the files you want to index. Change source and destination as necessary.

sudo docker-compose run --rm -v source:destination:ro shell bash

After above command finshes you'l be at the aleph docker terminal. At this point you can index the directory you want with this command. Where destination is the path you used in the previous command. Change the lang (-l) on the language of the files you want to index.

Note: Aleph uses Tesseract to do OCR.

aleph crawldir -l eng destination/some-directory

Depending on your system and size of data, indexing may take a long time. In the meantime you can view the status with the following commands.

View all datasets and investigations (known as collections).

aleph collections

Check status.

aleph status
By rbosaz, 13 February, 2023

I use this for my scanner.

Load:

pactl load-module module-loopback latency_msec=0

Unload:

pactl unload-module module-loopback

 

 

 

Tags

By rbosaz, 22 October, 2022

Note: This article is a compilation of notes taken while reading Linux Basics for Hackers by OccupyTheWeb. Commands apply to Kali Linux, but can be used on most Debian based distributions. Also, descriptions for most commands have come from the man pages. Enjoy!

Searching Linux

  • Basic
    • Locate: Will search the entire file system for your word(s). Unfortunately it uses an index/cache database that may not be up-to-date. The database can be updated manually via: sudo updatedb
  • Finding Binaries
    • whereis: This command returns location, source and man page, if available.
      • Only returns the location os binaries in the PATH variable. Typical directories include: /usr/bin and usr/sbin.
  • Advanced Searches:
    • find: Using a target location, find files base on filename, date of creation/modifcation, owner/group, permissions and size.
      • One can use wildcards:  * . ? and []
      • *: matches one or more occurrences of any character, including no character.
      • ?: matches a single occurrence of any character.
      • []: matches any occurrence of character enclosed in the square brackets. It is possible to use different types of characters (alphanumeric characters): numbers, letters, other special characters etc.
    • grep: A string and pattern matching utility that displays matching lines from multiple files.

File and Directory Administration

  • cat: Short for concatenation, used to display contents of a file, but can be used to create small files.
    • cat > test.txt: allows a user to start typing text into the file test.txt via interactive mode. Pressing ctrl+D will stop the capture. To append the same file replace the redirect (>) with a double redirect (>>).
  • touch: Originally created to change the date of a file (created or modified). If file doesn't exist will create an empty file.
  • mkdir: Creates directory.
  • cp: Creates a duplicate file in the new location
  • mv: Moves file from location to another. Also used to rename a file.
  • rm: Deletes a file. When used with the -r option will delete directory and all its contents.
  • rmdir: Removes directory only if contents are empty. Of course there's an option to ignore not empty messages.

View File's Contents

  • cat: Streams entire file contents to screen.
  • head: allows you to view the beginning of a file. Defaults to first 10 lines. Adding a dash (-) number will display those number of lines.
  • tail: Tail is the same as head except displays the last number of lines of a file.
  • nl: Displays a file with line numbers
  • grep: Allows one to filter the content of a file for display.
  • sed: Search file for occurrences of word or text pattern and then perform an action on it (i.e., replace). s/regexp/replacement/

Basic Linux Networking

  • ifconfig: basic tool to interact with active network interfaces. Ifconfig  is used to configure the kernel-resident network interfaces.  It is used at boot time to set up interfaces as necessary.  After that, it is usually only needed when debugging or when system tuning is needed.

    If no arguments are given, ifconfig displays the status of the currently active interfaces.  If a single interface argument is given,  it  displays  the  status of the given interface only; if a single -a argument is given, it displays the status of all interfaces, even those that are down.  Otherwise, it configures an interface.

    • Changing ip address of eth0: ifconfig eth0 <ip>

    • Changing ip, netmask and broadcast: sudo ifconfig eth0 <ip> netmask <ip> broadcast <ip>

    • Changing MAC Address: sudo ifconfig eth0 hw ether <address>

  • iwconfig: is  similar  to  ifconfig(8),  but is dedicated to the wireless interfaces. It is used to set the parameters of the network interface which are specific to the wireless operation (for example : the frequency).  Iwconfig may also be used to display  those  parameters,  and  the wireless statistics (extracted from /proc/net/wireless). All  these  parameters  and  statistics are device dependent. Each driver will provide only some of them depending on hardware support, and the range of values may change. Please refer to the man page of each device for details.
    • Understand this command becomes useful when using aircrack-ng. When cracking wireless passwords you'll want to put your interface in promiscuous mode.
  • dhclient: The Internet Systems Consortium DHCP Client, dhclient, provides a means for configuring one or more network interfaces using the  Dynamic  Host Configuration Protocol, BOOTP protocol, or if these protocols fail, by statically assigning an address.
    • Refreshing ip for eth0: dhclient eth0
  • dig: DNS lookup utility.

    dig is a flexible tool for interrogating DNS name servers. It performs DNS lookups and displays the answers that are  returned  from  the  name server(s)  that were queried. Most DNS administrators use dig to troubleshoot DNS problems because of its flexibility, ease of use, and clarity of output. Other lookup tools tend to have less functionality than dig.

    Although dig is normally used with command-line arguments, it also has a batch mode of operation for reading lookup requests  from  a  file.  A brief  summary of its command-line arguments and options is printed when the -h option is given. The BIND 9 implementation of dig allows multiple lookups to be issued from the command line.

    Unless it is told to query a specific name server, dig tries each of the servers listed in  /etc/resolv.conf. If no usable server addresses  are found, dig sends the query to the local host.

    When no command-line arguments or options are given, dig performs an NS query for "." (the root).

    It is possible to set per-user defaults for dig via ${HOME}/.digrc. This file is read and any options in it are applied before the command-line arguments. The -r option disables this feature, for scripts that need predictable behavior.

    The IN and CH class names overlap with the IN and CH top-level domain names. Either use the -t and -c options to specify the  type  and  class, use the -q to specify the domain name, or use "IN." and "CH." when looking up these top-level domains.

    • be aware that in Linux some users refer DNS as BIND (Berkeley Internet Name Domain). Same thing ...

  • Changing DNS server via /etc/resolv.conf

    • IF using a DHCP address and the server provides DNS. when renewing DHCP address will replace file contents.

  • Local DNS file with /etc/hosts

    • Ensure to use tab between ip and domain name, not space.

    • Structure is: ip <Tab> hostname  <Space> [aliases]...

Installing Software on Linux

CLI (via apt)

  • apt: provides a high-level commandline interface for the package management system. It is intended as an end user interface and enables some options better suited for interactive usage by default compared to more specialized APT tools like apt-get(8) and apt-cache(8).
  • apt search <package name>: can be used to search for the given regex(7) term(s) in the list of available packages and display matches. This can e.g. be useful if you are looking for packages having a specific feature. If you are looking for a package including a specific file try apt-file(1)

  • apt install, reinstall, remove, purge (apt-get(8)): Performs the requested action on one or more packages specified via regex(7), glob(7) or exact match. The requested action can be overridden for specific packages by appending a plus (+) to the package name to install this package or a minus (-) to remove it.

    A specific version of a package can be selected for installation by following the package name with an equals (=) and the version of the package to select. Alternatively the version from a specific release can be selected by following the package name with a forward slash (/)            and codename (bullseye, bookworm, sid ...) or suite name (stable, testing, unstable). This will also select versions from this release for dependencies of this package if needed to satisfy the request.

    Removing a package removes all packaged data, but leaves usually small (modified) user configuration files behind, in case the remove was an accident. Just issuing an installation request for the accidentally removed package will restore its function as before in that case. On
    the other hand you can get rid of these leftovers by calling purge even on already removed packages. Note that this does not affect any data or configuration stored in your home directory.

  • apt update: is used to download package information from all configured sources. Other commands operate on this data to e.g. perform package upgrades or search in and display details about all packages available for installation.

  • apt upgrade: is used to install available upgrades of all packages currently installed on the system from the sources configured via sources.list(5). New packages will be installed if required to satisfy dependencies, but existing packages will never be removed. If an         upgrade for a package requires the removal of an installed package the upgrade for this package isn't performed.

    • From time to time I'll execute sudo apt update && sudo apt upgrade -y to update and upgrade all in one shot. remove the -y if you're not feeling adventurous.

  • Repositories contain packages for particular Linux distros. To add or remove repositories edit this file: /etc/apt/sources.list. Here's a decent explanation of the file.

GUI (via apt)

  • Several "front-end" interfaces exist, such as aptitude(8), synaptic(8) and wajig(1). Depends on the Linux distribution you're using.

 

GitHub

When software is not available via a Linux repository, it may be available via GitHub. Do a search on GitHub. Copy the Code url and use the the following command  to clone the software to a location on your computer. Also, prior to cloning a project always read up on the project's documentation. I highly recommend creating a GitHub account and start using with scripts and software you create.

  • git clone <url>:

    Clones a repository into a newly created directory, creates remote-tracking branches for each branch in the cloned repository (visible using git branch --remotes), and creates and checks out an initial branch that is forked from the cloned repository’s currently active branch.

    After the clone, a plain git fetch without arguments will update all the remote-tracking branches, and a git pull without arguments will in addition merge the remote master branch into the current master branch, if any (this is untrue when "--single-branch" is given; see below).

Linux File System Permissions

  • ls -l: Long listing information about files and directories. For example ls -l /usr/share/hashcat provides the following listing:

       1                      2          3                     4              5                 6

   drwxr-xr-x 6 root root    103 Nov 13 13:43 charsets
   -rw-r--r-- 1 root root 240526 Jul 31  2020 hashcat.hcstat2
   -rw-r--r-- 1 root root  23281 Jul 31  2020 hashcat.hctune
   drwxr-xr-x 2 root root    140 Nov 13 13:43 layouts
   drwxr-xr-x 2 root root    340 Nov 13 13:43 masks
   lrwxrwxrwx 1 root root     25 Jul 31  2020 modules -> ../../lib/hashcat/modules
   drwxr-xr-x 2 root root  26286 Nov 13 13:43 OpenCL
   drwxr-xr-x 3 root root    681 Nov 13 13:43 rules
  1. first character is type (d = directory or - file) and the rest are the permissions in groups of three (rwx where r=read; w=write; x=execute). If a dash (-) exists that permissions is not granted. Grouping corresponds to owner (or user), group and other (all other users).
  2. number of links
  3. the user and group ownership
  4. size (bytes)
  5. created/modified date
  6. name (the above example also shows a symbolic link)       
  • chown: changes the user and/or group ownership of each given file.  If only an owner (a user name or numeric user ID) is given, that user is made the owner of each given file, and the files' group is not changed.  If the owner is followed by a colon and a group name (or numeric group ID), with no spaces between them, the group ownership of the files is changed as well.  If a colon but no group name follows the user name, that user is made the owner of the files and the group of the files is changed to that user's login group.  If the colon  and  group  are  given,  but  the owner is omitted, only the group of the files is changed; in this case, chown performs the same function as chgrp.  If only a colon is given, or if the entire operand is empty, neither the owner nor the group is changed.
  • chgrp: change group ownership. Change the group of each FILE to GROUP.  With --reference, change the group of each FILE to that of RFILE.
  • chmod: changes the file mode bits of each given file according to mode, which can be either a    symbolic representation of changes to make, or an octal number representing the bit pattern for the new mode bits.

    The format of a symbolic mode is [ugoa...][[-+=][perms...]...], where perms is either zero or more letters from the set rwxXst, or a  single  letter from the set ugo.  Multiple symbolic modes can be given, separated by commas.

    A  combination  of  the  letters  ugoa controls which users' access to the file will be changed: the user who owns it (u), other users in the file's group (g), other users not in the file's group (o), or all users (a).  If none of these are given, the effect is as if (a) were given, but bits that are set in the umask are not affected.

    The operator + causes the selected file mode bits to be added to the existing file mode bits of each file; - causes them to be removed; and = causes them to be added and causes unmentioned bits to be removed except that a directory's unmentioned set user and group ID bits are not affected.

    The letters rwxXst select file mode bits for the affected users: read (r), write (w), execute (or search for directories) (x),  execute/search  only if  the  file is a directory or already has execute permission for some user (X), set user or group ID on execution (s), restricted deletion flag or  sticky bit (t).  Instead of one or more of these letters, you can specify exactly one of the letters ugo: the permissions granted to  the  user  who owns  the file (u), the permissions granted to other users who are members of the file's group (g), and the permissions granted to users that are in neither of the two preceding categories (o).

    A numeric mode is from one to four octal digits (0-7), derived by adding up the bits with values 4, 2, and 1.  Omitted  digits  are  assumed  to  be leading zeros.  The first digit selects the set user ID (4) and set group ID (2) and restricted deletion or sticky (1) attributes.  The second digit
    selects permissions for the user who owns the file: read (4), write (2), and execute (1); the third selects  permissions  for  other  users  in  the file's group, with the same values; and the fourth for other users not in the file's group, with the same values.

    chmod  never changes the permissions of symbolic links; the chmod system call cannot change their permissions.  This is not a problem since the permissions of symbolic links are never used.  However, for each symbolic link listed on the  command  line,  chmod  changes  the  permissions  of  the pointed-to file.  In contrast, chmod ignores symbolic links encountered during recursive directory traversals.

  • Linux set permission defaults for files and directories as 666 and 777 respectively. But Debian systems umask is preconfigured to 022. umask subtracts from the default, so 666 becomes 644 and 777 becomes 755. Each user may modify their umask via ~/.profile file.

  • SUID (Set owner User ID up on execution) is a special type of file permissions given to a file. Users will get file owner’s permissions as well as owner UID and GID when executing a file/program/command. 

    • To SUID using symbolic: chmod u+s <file name>

    • To SUID using numeric (octal): chmod 4750 <file name>

    • When a file has its SUID bit set long listing permissions will like: -rwsr-x---

  • SGID (Set Group ID up on execution) is a special type of file permissions given to a file/folder. Users will get file Group’s permissions when executing a Folder/file/program/command. When the bit is set on a directory new files created in that directory goes to the directory creator's group.

    • To SGID using symbolic: chmod g+s <file name>

    • To SGID using numeric (octal): chmod 2750 <file name>

    • When a file has its SUID bit set long listing permissions will like: -rwxr-s---

  • Sticky bit is a permision bit set on a directory, allowing a user to rename or delete files contained in it. This is a legacy function and not used anymore.

  • Exploiting SUID and SGID: Sometimes a developer may set the SUID bit of a program to access files that require root privileges. This allows a user to gain unintentional root privileges, providing the user do whatever they want. the following command will list all filesfor the root user with the permissions 4000: find / -user root -perm -4000

    • Similarly this can be done for SGID. Change -4000 to -2000 to find files/directories

By rbosaz, 22 September, 2022

After getting my amateur radio licenses (over a year ago), I switched from Linux to Windows because most software for my radios seem to only target Windows. Prior to the switch, I used Linux as my main computers (laptops and servers) for over 15 years, so this was done under extreme protest. How has it been going?

HORRIBLE!

By rbosaz, 11 September, 2022

I'm running a variety of Ubuntu Linux distribution known as Pop_OS! and have had several issues when for some reason or another Proton VPN is not shutdown gracefully leaving me without internet but still having LAN access.

Proton VPN appears to install an IPV6 "leak" interface which seems to be the problem.

The following instructions will usually bring back an internet connection..