Creating and Configuring an Amazon EC2 AMI OSF Instance

Introduction
This documentation page outline all the steps required to create and configure a vanilla Open Semantic Framework version 3.1 EC2 AMI instance. This includes the configuration of all the users, firewall and security settings, along with the creation of a new, non-vanilla, OSF network.

Creating a New OSF EC2 Instance
The first step is to create a new instance Amazon EC2 instance. If you are not familiar with Amazon EC2, you should read the Getting Started with Amazon EC2 Linux Instances guide. The one thing you have to do in the Launch an Amazon EC2 Instance section is to click the "Community AMIs" left tab, and then to search for the AMI ID listed below. Once you see it appearing, you will be able to click the  button to start creating the OSF instance.

Available AMIs are:

Configure the OSF Instance
Once you created the new OSF instance, and once you logged into your newly created instance, you have to configure it such that it is secure and that it works using the domain name of your choice.

Configure Ubuntu Users
The first step is to create the users that will be used to access the OSF server and to properly configure the SSH daemon.

Create New Administrator User Account
The goal is to create a new administrator user account that is not any default users on a Ubuntu server.

Key Generation and Distribution
This is how to create similar keypairs for all users who need access to your instances.

In the example below (to be run on your local machine, not your EC2 instance) replace "user" with the actual user's login, name or some other unique identifier.
 * 1) cd /tmp
 * 2) ssh-keygen -b 1024 -f user -t dsa

This will create 2 files:
 * user (private key)
 * user.pub (public key)Copy all the public key files that you generated to a temporary place on your instance:
 * 1) scp -i root *.pub ec2-your-instance-name.compute.amazonaws.com:/tmp

Administrator User Account Creation
Log in to the instance as root. For each user you are creating, add the user to your instance with the
 * 1) useradd -m -c "firstname lastname" user

For simplicity's sake, use the same "user" name as you did for key generation. Now we need to place the key into their ssh authorized keys file (again, replacing "user" with the username you chose earlier) Finally create a new password for that user:
 * 1) cd ~user
 * 2) mkdir .ssh
 * 3) chmod 700 .ssh
 * 4) chown user:user .ssh
 * 5) cat /tmp/user.pub >> .ssh/authorized_keys
 * 6) chmod 600 .ssh/authorized_keys
 * 7) chown user:user .ssh/authorized_keys
 * 1) passwd user

Make the User Sudoer
This step is optional. You only perform these steps if you want the new user to be a sudoer. After the line: root    ALL=(ALL:ALL) ALL add the line: user    ALL=(ALL:ALL) ALL Finally save the file.
 * 1) vim /etc/sudoers

Disable Password-based Login
Log in to your instance as root and edit the ssh daemon configuration file:


 * 1) vim /etc/ssh/sshd_config


 * 1) vim /etc/ssh/sshd_config

find the line:

PermitRootLogin without-password

and add the AllowUsers entry and change PermitRootLogin to:

PermitRootLogin no AllowUsers user

Again, be sure that you have an active login, save the file and restart sshd:


 * 1) service ssh restart

Delete Default Ubuntu User
Finally let's delete the default ubuntu user.

First, log into another shell '''without closing the one you are connected to with the ubuntu user. '''Once you validate that you can log into the instance using the admin user you just created above, to close the other shell terminal where you are logged in using the  user. Then delete the ubuntu user:


 * 1) userdel -r ubuntu

Configure Firewall
The next step is to properly configure the firewall such that only the necessary ports are open and available on the Internet. ufw allow ssh/tcp ufw allow 80/tcp ufw allow proto tcp from 127.0.0.1 to any port 8890 ufw allow proto tcp from 127.0.0.1 to any port 8983 ufw logging on ufw enable ufw status Here is the explanation of each of these commands: Additionally, you can add more ports and IP addresses using modified versions of these commands.
 * 1) Enable everyone to send queries to the port 22
 * 2) Enable everyone to send queries to the port 80
 * 3) Enable the localhost to send queries on the port 8890 (Virtuoso)
 * 4) Enable the localhost to send queries on the port 8983 (Solr)
 * 5) Enable firewall logging
 * 6) Enable firewall
 * 7) Get the status of the firewall and make sure it is properly running

Delete Ontologies
We have to delete the ontologies before re-creating the OSF network that uses the new domain name. You can delete them by using the following commands: omt --delete="file://localhost/data/ontologies/files/rdf.xml" --osf-web-services=" http://localhost/ws/ " omt --delete="file://localhost/data/ontologies/files/owl.rdf" --osf-web-services=" http://localhost/ws/ " omt --delete="file://localhost/data/ontologies/files/rdfs.xml" --osf-web-services=" http://localhost/ws/ " omt --delete="file://localhost/data/ontologies/files/aggr.owl" --osf-web-services=" http://localhost/ws/ " omt --delete="file://localhost/data/ontologies/files/iron.owl" --osf-web-services=" http://localhost/ws/ " omt --delete="file://localhost/data/ontologies/files/sco.owl" --osf-web-services=" http://localhost/ws/ " omt --delete="file://localhost/data/ontologies/files/wgs84.owl" --osf-web-services=" http://localhost/ws/ " omt --delete="file://localhost/data/ontologies/files/wsf.owl" --osf-web-services=" http://localhost/ws/ "

Change Default Passwords
The vanilla Opens Semantic Framework AMI is packed with default passwords. This section covers all the places where default passwords need to be modified.

MySQL
To change the password of the  user, you have to: The default password of the  user is "root".
 * 1) mysqladmin -u root -p'root' password NEW-PASSWORD

PHPMyAdmin
To change the password of the  user, you have to log into MySQL using the   user: Then run the following SQL command to change the default password of the  user. Note: you have to modify that SQL command to change the password SET PASSWORD FOR 'phpmyadmin'@'localhost' = PASSWORD(' NEW-PASSWORD '); The default password of the  user is "root".
 * 1) mysql -u root -p

Virtuoso
To change the password of the and the  users, you have to: Then, in the  command line tool, once you are logged-in, to type the following commands: set password dba NEW-DBA-PASSWORD; update DB.DBA.SYS_USERS set U_PASSWORD=' NEW-DAV-PASSWORD ' where U_NAME='dav'; The default password of the  and   users is "dba".
 * 1) /usr/bin/isql-vt

Memcached
To change the password of the  user, you have to:
 * 1) sed -i "s>define('ADMIN_PASSWORD','admin');>define('ADMIN_PASSWORD','NEW-PASSWORD');>" /usr/share/memcached-ui/index.php

The default password of the  is "admin".

Reconfigure the Open Semantic Framework
Now that we changed all the default passwords, we have to re-configure them into the OSF instance. To reconfigure it, you to: The first thing you have to modify is the URL of the OSF instance on the web. By default, it is defined as. To change it, search for the following lines, and update the  setting accordingly:
 * 1) vim /data/osf-web-services/configs/osf.ini

[network] wsf_base_url = " http://localhost "

Then you have to update the default WSF graph URI. It should be using the same domain that you defined above:

[datasets] wsf_graph = " http://localhost/wsf/ "

Finally you have to update the password of the Virtuoso server as you defined it for the  user above:

[triplestore] password = "dba"

Reconfigure API Key
You have to create a new API Key for the  Application ID. You can generate a unique 32 characters API Key by using the following command: Then run the following command to change the API key you just created into the registry of API Keys:
 * 1) php -r 'echo "\n\n".strtoupper(bin2hex(openssl_random_pseudo_bytes(16)))."\n\n";'
 * 1) sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /data/osf-web-services/configs/keys.ini

Update API Key References
Once the API Key is updated, we have to update its references in few different configuration files. Run the following commands to update the API Key setting with the new key:


 * 1) sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /usr/share/ontologies-management-tool/omt.ini
 * 2) sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /usr/share/datasets-management-tool/dmt.ini
 * 3) sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /usr/share/permissions-management-tool/pmt.ini
 * 4) sed -i "s>E74ACB52F3F0E54764C786BFBB438E4E>NEW-API-KEY>" /usr/share/osf/StructuredDynamics/osf/tests/Config.php

Then edit the domain reference in the same files:


 * 1) sed -i "s>localhost>OSF-NETWORK-DOMAIN>" /usr/share/ontologies-management-tool/omt.ini
 * 2) sed -i "s>localhost>OSF-NETWORK-DOMAIN>" /usr/share/datasets-management-tool/dmt.ini
 * 3) sed -i "s>localhost>OSF-NETWORK-DOMAIN>" /usr/share/permissions-management-tool/pmt.ini
 * 4) sed -i "s> http://localhost >http://OSF-NETWORK-DOMAIN>" /usr/share/osf/StructuredDynamics/osf/tests/Config.php

Delete Vanilla Network
Log into Virtuoso to delete the vanilla network: Then once you are logged into Virtuoso, run the following command: sparql clear graph < http://localhost/wsf/ >;
 * 1) /usr/bin/isql-vt

Recreate Network
To re-create the network using the new setup, we will use one of the script used by the OSF Installer to create the network. Run the following set of commands, and properly specify the information in bold:


 * 1) cp /usr/share/osf-installer/resources/virtuoso/initialize_osf_web_services_network.php /tmp/
 * 2) sed -i "s>\"dba\", \"dba\">\"dba\", \"VIRTUOSO-DBA-PASSWORD\">" /tmp/initialize_osf_web_services_network.php
 * 3) sed -i "s>$server_address = \" http://localhost \";>$server_address = \"http://OSF-NETWORK-DOMAIN\";>" /tmp/initialize_osf_web_services_network.php
 * 4) php /tmp/initialize_osf_web_services_network.php
 * 5) rm /tmp/initialize_osf_web_services_network.php

Re-import the Ontologies
Now that the OSF network is re-created, we have to re-import the core ontologies:
 * 1) omt --load-advanced-index="true" --load-all --load-list="/usr/share/osf-installer/resources/osf-web-services/ontologies.lst" --osf-web-services="http://OSF-NETWORK-DOMAIN/ws/"

Run Tests Suites
Now that we re-configured a vanilla EC2 OSF instance, we will rerun the OSF Web Services Tests Suites to make sure that everything is still properly working with all the new settings. To run the tests suites, you have to perform the following commands:

cd /usr/share/osf/StructuredDynamics/osf/tests/ phpunit --configuration phpunit.xml --verbose --colors --log-junit log.xml

If all the tests pass, it means that your new OSF instance is properly re-configured.

Install OSF for Drupal
An additional step you can do is to install OSF for Drupal on that new server, or on any other servers that will use this new OSF instance:


 * 1) sed -i "s>localhost>OSF-NETWORK-DOMAIN>" /usr/share/osf-installer/installer.ini
 * 2) cd /usr/share/osf-installer/
 * 3) ./osf-installer --install-osf-drupal