Creating and Configuring an Amazon EC2 AMI OSF Instance
- 1 Introduction
- 2 Creating a New OSF EC2 Instance
- 3 Configure the OSF Instance
- 3.1 Make yourself root
- 3.2 Configure Ubuntu Users
- 3.2.1 Create New Administrator User Account
- 3.2.2 Disable Password-based Login
- 3.2.3 Delete Default Ubuntu User
- 3.2.4 Configure Firewall
- 3.2.5 Delete Ontologies
- 3.2.6 Change Default Passwords
- 3.2.7 Reconfigure the Open Semantic Framework
- 3.2.8 Reconfigure API Key
- 3.2.9 Re-create the OSF Network
- 3.2.10 Re-import the Ontologies
- 3.2.11 Run Tests Suites
- 3.2.12 Install OSF for Drupal
This documentation page outline all the steps required to create and configure a vanilla Open Semantic Framework version 3.1 EC2 AMI instance. This includes the configuration of all the users, firewall and security settings, along with the creation of a new, non-vanilla, OSF network.
Creating a New OSF EC2 Instance
The first step is to create a new instance Amazon EC2 instance. If you are not familiar with Amazon EC2, you should read the Getting Started with Amazon EC2 Linux Instances guide. The one thing you have to do in the Launch an Amazon EC2 Instance section is to click the "Community AMIs" left tab, and then to search for the AMI ID listed below. Once you see it appearing, you will be able to click the
Select button to start creating the OSF instance.
Available AMIs are:
|Region||arch||root store||OSF Version||Ubuntu Version||AMI|
Configure the OSF Instance
Once you created the new OSF instance, and once you logged into your newly created instance, you have to configure it such that it is secure and that it works using the domain name of your choice.
Make yourself root
The first step is to make yourself root before executing the commands outlined below:
# sudo -i
Configure Ubuntu Users
The first step is to create the users that will be used to access the OSF server and to properly configure the SSH daemon.
Create New Administrator User Account
The goal is to create a new administrator user account that is not any default users on a Ubuntu server.
Key Generation and Distribution
This is how to create similar keypairs for all users who need access to your instances.
In the example below (to be run on your local machine, not your EC2 instance) replace "user" with the actual user's login, name or some other unique identifier.
# cd /tmp # ssh-keygen -b 1024 -f user -t dsa
This will create 2 files:
- user (private key)
- user.pub (public key)Copy all the public key files that you generated to a temporary place on your instance:
# scp -i root *.pub ec2-your-instance-name.compute.amazonaws.com:/tmp
Administrator User Account Creation
Log in to the instance as root. For each user you are creating, add the user to your instance with the
# useradd -m -c "firstname lastname" user
For simplicity's sake, use the same "user" name as you did for key generation. Now we need to place the key into their ssh authorized keys file (again, replacing "user" with the username you chose earlier)
# cd ~user # mkdir .ssh # chmod 700 .ssh # chown user:user .ssh # cat /tmp/user.pub >> .ssh/authorized_keys # chmod 600 .ssh/authorized_keys # chown user:user .ssh/authorized_keys
Finally create a new password for that user:
# passwd user
If you want to use the
bash shell by default for that new user, just run the following command:
# chsh -s /bin/bash user
Make the User Sudoer
This step is optional. You only perform these steps if you want the new user to be a sudoer.
# vim /etc/sudoers
After the line:
root ALL=(ALL:ALL) ALL
add the line:
user ALL=(ALL:ALL) ALL
Finally save the file.
Disable Password-based Login
Log in to your instance as root and edit the ssh daemon configuration file:
# vim /etc/ssh/sshd_config
# vim /etc/ssh/sshd_config
find the line:
and add the AllowUsers entry and change PermitRootLogin to:
PermitRootLogin no AllowUsers user
Again, be sure that you have an active login, save the file and restart sshd:
# service ssh restart
Delete Default Ubuntu User
Finally let's delete the default ubuntu user.
First, log into another shell without closing the one you are connected to with the ubuntu user. Once you validate that you can log into the instance using the admin user you just created above, to close the other shell terminal where you are logged in using the
ubuntu user. Then delete the ubuntu user:
# userdel -r ubuntu
The next step is to properly configure the firewall such that only the necessary ports are open and available on the Internet.
ufw allow ssh/tcp
ufw allow 80/tcp
ufw allow proto tcp from 127.0.0.1 to any port 8890
ufw allow proto tcp from 127.0.0.1 to any port 8983
ufw logging on
Here is the explanation of each of these commands:
- Enable everyone to send queries to the port 22
- Enable everyone to send queries to the port 80
- Enable the localhost to send queries on the port 8890 (Virtuoso)
- Enable the localhost to send queries on the port 8983 (Solr)
- Enable firewall logging
- Enable firewall
- Get the status of the firewall and make sure it is properly running
Additionally, you can add more ports and IP addresses using modified versions of these commands.
We have to delete the ontologies before re-creating the OSF network that uses the new domain name. You can delete them by using the following commands:
# omt --delete --osf-web-services="http://localhost/ws/"
Then delete all the ontologies that appears in the list of loaded ontologies.
Change Default Passwords
The vanilla Opens Semantic Framework AMI is packed with default passwords. This section covers all the places where default passwords need to be modified.
To change the password of the
dba and the
dav users, you have to:
Then, in the
isql command line tool, once you are logged-in, to type the following commands:
set password dba NEW-DBA-PASSWORD;
update DB.DBA.SYS_USERS set U_PASSWORD='NEW-DAV-PASSWORD' where U_NAME='dav';
The default password of the
dav users is "dba".
To change the password of the
admin user, you have to:
# sed -i "s>define('ADMIN_PASSWORD','admin');>define('ADMIN_PASSWORD','NEW-PASSWORD');>" /usr/share/memcached-ui/index.php
The default password of the
admin is "admin".
Reconfigure the Open Semantic Framework
Now that we changed all the default passwords, we have to re-configure them into the OSF instance. To reconfigure it, you to:
# vim /data/osf-web-services/configs/osf.ini
The first thing you have to modify is the URL of the OSF instance on the web. By default, it is defined as
localhost. To change it, search for the following lines, and update the
wsf_base_url setting accordingly:
[network] wsf_base_url = "http://localhost"
Then you have to update the default WSF graph URI. It should be using the same domain that you defined for the
[datasets] wsf_graph = "http://localhost/wsf/"
Finally you have to update the password of the Virtuoso server as you defined it for the
dba user above:
[triplestore] password = "dba"
Reconfigure API Key
You have to create a new API Key for the
administer Application ID. You can generate a unique 32 characters API Key by using the following command:
# php -r 'echo "\n\n".strtoupper(bin2hex(openssl_random_pseudo_bytes(16)))."\n\n";'
Then run the following command to change the API key you just created into the registry of API Keys:
# sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /data/osf-web-services/configs/keys.ini
Update API Key References
Once the API Key is updated, we have to update its references in few different configuration files. Run the following commands to update the API Key setting with the new key:
# sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /usr/share/ontologies-management-tool/omt.ini # sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /usr/share/datasets-management-tool/dmt.ini # sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /usr/share/permissions-management-tool/pmt.ini # sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /usr/share/osf/StructuredDynamics/osf/tests/Config.php
Then edit the domain reference in the same files:
# sed -i "s>localhost>OSF-NETWORK-DOMAIN>" /usr/share/ontologies-management-tool/omt.ini # sed -i "s>localhost>OSF-NETWORK-DOMAIN>" /usr/share/datasets-management-tool/dmt.ini # sed -i "s>localhost>OSF-NETWORK-DOMAIN>" /usr/share/permissions-management-tool/pmt.ini # sed -i "s>http://localhost>http://OSF-NETWORK-DOMAIN>" /usr/share/osf/StructuredDynamics/osf/tests/Config.php
Re-create the OSF Network
Delete Vanilla Network
Log into Virtuoso to delete the vanilla network:
Then once you are logged into Virtuoso, run the following command:
sparql clear graph <http://localhost/wsf/>;
To re-create the network using the new setup, we will use one of the script used by the OSF Installer to create the network. Run the following set of commands, and properly specify the information in bold:
# cp /usr/share/osf-installer/resources/virtuoso/initialize_osf_web_services_network.php /tmp/ # sed -i "s>\"dba\", \"dba\">\"dba\", \"VIRTUOSO-DBA-PASSWORD\">" /tmp/initialize_osf_web_services_network.php # sed -i "s>$server_address = \"http://localhost\";>$server_address = \"http://OSF-NETWORK-DOMAIN\";>" /tmp/initialize_osf_web_services_network.php # php /tmp/initialize_osf_web_services_network.php # rm /tmp/initialize_osf_web_services_network.php # /usr/bin/isql-v 1111 dba VIRTUOSO-DBA-PASSWORD /tmp/init_osf.sql # rm /tmp/init_osf.sql
Re-import the Ontologies
Now that the OSF network is re-created, we have to re-import the core ontologies:
# omt --load-advanced-index="true" --load-all --load-list="/usr/share/osf-installer/resources/osf-web-services/ontologies.lst" --osf-web-services="http://OSF-NETWORK-DOMAIN/ws/"
Run Tests Suites
Now that we re-configured a vanilla EC2 OSF instance, we will rerun the OSF Web Services Tests Suites to make sure that everything is still properly working with all the new settings. To run the tests suites, you have to perform the following commands:
# cd /usr/share/osf/StructuredDynamics/osf/tests/ # phpunit --configuration phpunit.xml --verbose --colors --log-junit log.xml
If all the tests pass, it means that your new OSF instance is properly re-configured.
Install OSF for Drupal
An additional step you can do is to install OSF for Drupal on that new server, or on any other servers that will use this new OSF instance:
# sed -i "s>localhost>OSF-NETWORK-DOMAIN>" /usr/share/osf-installer/installer.ini # cd /usr/share/osf-installer/ # ./osf-installer --install-osf-drupal