Build A Custom Chatbot Using Python (Custom Knowledge Base)

What Is A Vector Database? What is a Vector? What are “Vector Embeddings”?

In order to do that, we are going to use the sentence-transformers library from Hugging Face. Here is a link to the official docs.

So, in order to install the library you have to do something like:

> pip install -U sentence-transformers

And then you can use the library in a script like:

from sentence_transformers import SentenceTransformer, util

input_sentence = "That is a happy person."

sentences_to_compare = [
    "That is a happy dog",
    "That is a very happy person",
    "She is sad and lonely. Especially after her cat died.",
    "Pharrell Williams wrote the song 'Happy' in 2014",
    "Delhi is the capital of India",
]

model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')

input_embedding = model.encode(input_sentence)

print("Input Sentence:", input_sentence, "\n")

# Next we loop over all the sentences we want to compare 
# and print the similarity between the input sentence and 
# the sentence we are comparing it to.
for sentence in sentences_to_compare:
    sentence_embedding = model.encode(sentence)
    similarity_tensor = util.pytorch_cos_sim(input_embedding, sentence_embedding)[0][0]
    similarity_value = similarity_tensor.item()
    print("Sentence:", sentence)
    print("Similarity:", similarity_value)
    print("===")

In the above script, we are using the model named: all-MiniLM-L6-v2

If you run the above script, you will get a response like:

Input Sentence: That is a happy person. 

Sentence: That is a happy dog
Similarity: 0.671120285987854
===
Sentence: That is a very happy person
Similarity: 0.9342775940895081
===
Sentence: She is sad and lonely. Especially after her cat died.
Similarity: 0.2835640013217926
===
Sentence: Pharrell Williams wrote the song 'Happy' in 2014
Similarity: 0.3601309061050415
===
Sentence: Delhi is the capital of India
Similarity: 0.007062782533466816
===

As you can see the “similarity” number is high when the sentence is like the original sentence. And it’s low when the sentence is different. The model does make mistakes too. (I added the sentence about the song “Happy” to show that.) But still, it’s pretty decent. It surely looked at the last sentence and noticed that it is completely different from the others.

Note: The first time you run this, you might notice that the model takes some time to download. But that does not happen next time you run it. It’s just a one-time thing.

Another thing to note about the embeddings is that they are in the form of arrays that are 384 elements long. You can inspect the array by running something like:

from sentence_transformers import SentenceTransformer, util

sentence = "That is a happy person."
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
embedding = model.encode(sentence)

print("Length Of The Vector:", len(embedding))
print("Vector:", embedding)

When you run the above script, you will get something like:

Length Of The Vector: 384

Vector: [-2.46720240e-02  8.61168057e-02  4.57021147e-02 -4.09682579e-02
 -6.84659779e-02 -7.66141200e-03  1.18745767e-01 -8.63582268e-03
  3.67808677e-02 -1.06625259e-02  8.19263831e-02 -4.80930554e-03
 -1.19793769e-02 -1.00510260e-02  2.45560128e-02  2.30727140e-02
 -6.91623241e-02 -5.53910844e-02  3.22435275e-02  1.60760200e-03
 -7.78550059e-02 -7.77970860e-03 -4.07488365e-03  1.37812346e-02
  3.83134466e-03 -1.71690937e-02  3.16330455e-02 -7.38037284e-04
  6.19726591e-02  6.10892773e-02 -3.32377814e-02 -2.52756327e-02
  9.37762111e-02  2.06878968e-02 -3.24028656e-02  3.10536362e-02
 -2.46816631e-02  1.35345971e-02 -1.00918012e-02  2.36975066e-02
 -9.18709673e-03  2.17065495e-02  6.77749701e-03  2.47748215e-02
 -1.86500996e-02 -4.49814014e-02  6.29933551e-02 -2.55798511e-02
  8.76966417e-02 -3.07950340e-02 -1.20663848e-02  4.39662598e-02
 -5.97078018e-02 -7.29687586e-02  7.14917630e-02  5.20428605e-02
  1.00254148e-01 -2.43245140e-02  3.77747826e-02  6.29629046e-02
  2.11970154e-02  7.01688826e-02 -5.49402237e-02  8.24060917e-05
  5.65452501e-02  1.18269827e-02 -3.87434699e-02 -3.22240181e-02
 -1.89375002e-02 -1.02068372e-02 -4.06253226e-02  4.63818945e-02
  7.90664405e-02  7.96727464e-03 -1.78388264e-02 -2.79942751e-02
  4.97788303e-02  4.17374074e-02 -2.57583172e-03  1.18747436e-01
  5.00707477e-02 -8.46506581e-02  3.41651542e-03 -4.55199182e-02
 -1.98047683e-02 -2.80259382e-02 -7.09016714e-03 -2.53995024e-02
 -8.73575686e-04  6.68466964e-04 -5.37528209e-02 -5.45573467e-03
  5.10725901e-02 -5.68201803e-02 -7.56285116e-02 -6.81172609e-02
 -6.01926595e-02  1.66541226e-02 -6.97019398e-02  5.28640151e-02
 -3.15990075e-02  2.70118825e-02  6.91894293e-02  3.89907807e-02
  2.59317458e-02  3.61612029e-02 -4.51193824e-02  4.85728867e-02
  1.30623095e-02 -7.77506381e-02 -7.20117018e-02 -1.97788682e-02
  4.79546040e-02  3.84162217e-02  2.95297876e-02  6.85002059e-02
 -1.00446440e-01  5.86183853e-02  2.03515347e-02 -4.56160270e-02
  2.58496739e-02  3.51988710e-02  6.07746914e-02  3.31665948e-02
  1.01267640e-02 -8.02919716e-02  2.76818778e-02 -7.61296794e-33
  8.64675175e-03  4.12631184e-02  5.49233183e-02  5.05420030e-04
 -1.53608527e-03  3.26667354e-02 -5.71187548e-02 -5.83009832e-02
 -3.72897014e-02  1.56709068e-02 -1.44537792e-01 -2.17781942e-02
 -8.87899101e-02 -2.93913838e-02 -7.70181566e-02 -4.08902168e-02
 -6.91767633e-02  3.31365056e-02 -6.48183748e-02  4.33204025e-02
 -6.24118932e-02 -2.08823532e-02  3.80173475e-02 -1.31444074e-02
 -8.77470057e-03  6.37468242e-04  1.04280062e-01 -1.76669452e-02
  4.22190912e-02 -1.05489034e-03 -9.12069082e-02  5.35855144e-02
  3.51020359e-02 -5.98102026e-02 -4.18799371e-02 -1.18005753e-01
 -1.23558715e-02 -5.46073578e-02 -6.66720122e-02 -1.71420760e-02
 -2.89418958e-02  3.53698060e-02  3.42779979e-03  6.78499341e-02
 -6.93865865e-02  3.12773064e-02  5.48955426e-02 -3.69304419e-03
  5.98255917e-02 -4.72623780e-02  3.08180284e-02 -3.50813270e-02
  6.77209124e-02 -7.12634623e-02 -3.98674235e-02 -4.70543317e-02
  5.69201112e-02  3.03146429e-02  6.68351129e-02  3.61000821e-02
 -6.00113347e-03  3.18250209e-02 -1.21381260e-01 -1.28578888e-02
  3.69424745e-02  3.87968197e-02  2.92852800e-02  1.54049713e-02
  6.07613511e-02  4.13324125e-02  4.94139716e-02  6.69576898e-02
  5.47784306e-02 -5.14714383e-02 -1.02930076e-01  1.76078584e-02
 -1.61633193e-02 -7.54488185e-02 -3.15386336e-03 -1.29731763e-02
  7.22425133e-02 -1.51209412e-02 -1.37219662e-02 -1.85537674e-02
  2.41122209e-02 -1.00316098e-02 -1.71473101e-02 -5.80143146e-02
 -8.99287462e-02  6.33649305e-02  1.15514420e-01  1.77180395e-02
  6.21020012e-02 -1.70754157e-02 -6.34599384e-03  5.32048727e-33
  2.11590435e-02  1.71505846e-02 -2.87544690e-02  5.75419404e-02
  1.62429791e-02 -1.92669444e-02 -2.76364498e-02  7.09576383e-02
 -8.90385211e-02  9.32738483e-02  1.10832401e-01 -4.52367216e-02
  9.41417068e-02  2.91315038e-02  9.09692142e-03  4.75034341e-02
  3.39463949e-02  1.81509983e-02 -1.09885251e-02  4.03245501e-02
 -3.28887887e-02  1.12904027e-01 -2.82317083e-02  6.47905320e-02
 -3.08069158e-02  5.25683165e-02  6.18184209e-02 -8.52767229e-02
 -5.27298935e-02 -4.34465557e-02 -1.92130022e-02  3.57361138e-02
 -7.64198527e-02 -1.43524483e-01  3.52843814e-02 -2.08135974e-02
  2.43109278e-02  4.33928743e-02 -2.91502383e-02 -1.04186721e-02
  2.40339134e-02  2.00014357e-02  3.01263481e-02  3.89626436e-02
  5.74229565e-03  1.95347387e-02  8.24517235e-02 -5.96813187e-02
  1.77431609e-02  9.75705385e-02 -4.40084822e-02 -3.62145379e-02
  2.45158412e-02  5.30302487e-02  1.69978011e-02 -2.45407317e-02
  1.72837789e-03 -7.81385079e-02  5.21761924e-02 -6.37797490e-02
 -1.23364270e-01 -5.13468310e-03  4.37647849e-02  3.83304693e-02
  4.66830470e-02 -3.49097922e-02 -9.71097201e-02 -5.60870999e-03
 -5.49964532e-02  1.71237420e-02  7.84234032e-02 -6.95970133e-02
 -6.35316372e-02  4.50071041e-03 -1.18954359e-02 -5.62967584e-02
  3.74892424e-03 -5.25021087e-03  7.17931092e-02 -5.74724786e-02
 -5.48331998e-02 -4.32234863e-03  6.39331192e-02  4.20721658e-02
 -1.35933105e-02 -6.50977716e-02 -3.42731550e-03 -4.04644012e-02
  8.28365679e-04  5.05046993e-02  3.36399898e-02  1.57138519e-02
 -3.89398634e-02 -3.00349165e-02 -1.16703063e-02 -1.95086454e-08
 -2.49570236e-02 -5.38664497e-02 -7.28454068e-02 -7.39433989e-02
  4.41159979e-02  2.32290085e-02  1.01542197e-01 -8.27160329e-02
 -3.71212400e-02  5.08565716e-02  3.76091860e-02  5.65130748e-02
  9.05696675e-03 -5.14653251e-02  2.75405906e-02 -5.43971593e-03
  1.17480658e-01 -1.46246990e-02 -1.25490678e-02  9.19553190e-02
  7.86352083e-02  2.07750872e-03  3.11274976e-02  9.44895819e-02
 -7.27683902e-02  3.50676687e-03 -7.61597008e-02 -3.98025401e-02
 -4.48514558e-02  3.95999439e-02  1.55153703e-02 -4.16716095e-03
 -3.80127914e-02  9.61000752e-03  1.03350855e-01  7.62674287e-02
  1.24495178e-02 -2.60304958e-02  3.90044004e-02  9.18273181e-02
 -6.38172850e-02  6.21212013e-02 -7.44379461e-02  8.65186751e-02
 -3.80681083e-02 -3.03872973e-02  3.93028818e-02 -5.21965101e-02
 -1.87056083e-02  3.73292305e-02  7.71731585e-02  6.29260996e-03
 -3.28463279e-02  8.04478377e-02 -3.20375860e-02 -1.11916132e-01
  4.59426753e-02  1.29644442e-02 -4.61628065e-02 -5.43179875e-03
 -2.20686086e-02 -2.13993490e-02 -2.96242088e-02  3.77730653e-02]

So, once the sentence is converted into a vector, how do we find which ones are similar exactly? Cosine Similarity!!

If we are comparing 2 vectors and we want to know how similar they are, we can do that very fast using something called “cosine similarity”.  Below is a nice video that explains this topic in some detail:

Things in the video have been simplified quite a bit. But, the same math and principles are used in our vectors above.

The “util” module used in the above script provides a quick and easy way to find similarities using the above-explained method. Below I am putting a part of the script again for the sake of clarity:

for sentence in sentences_to_compare:
    sentence_embedding = model.encode(sentence)
    # Below is where we do the actual comparison using the cosine 
    # similarity function from the sentence_transformers library.
    similarity_tensor = util.pytorch_cos_sim(input_embedding, sentence_embedding)[0][0]
    similarity_value = similarity_tensor.item()
    print("Sentence:", sentence)
    print("Similarity:", similarity_value)
    print("===")

Note That: The above code is just for demonstration. In order to find the quotes that are similar to our question, in the real project, we are going to make use of the Vector Database Pinecone. It is going to handle the storing of the data and the querying of the data based on cosine similarity.

> pip install pinecone-client

import json

# Import the SentenceTransformer model
from sentence_transformers import SentenceTransformer

# Import Pinecone
import pinecone
import hashlib

# Initialize Pinecone and connect to the "quotes" index
pinecone.init(api_key="PUT-YOUR-PINE-CODE-API-KEY-HERE", environment="us-west4-gcp")
index = pinecone.Index("NAME-OF-THE-INDEX-YOU-CREATED")

# Set up the model
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')

# Read the quotes from the JSON file
file = open('quotes.json', 'r')
file_contents_json = file.read()

# Parse the JSON file
contents_array = json.loads(file_contents_json)

# Loop over the quotes and add them to the index
for quote in contents_array:
    # Extract the quote & author
    the_quote = quote['Quote']
    author = quote['Author']
    
    # Generate an MD5 hash of the quote
    # This will be used as the unique ID for the quote
    # in the Pinecone index
    md5_hash = hashlib.md5(the_quote.encode()).hexdigest()

    # Encode the quote using the model
    embedding = model.encode(the_quote)
    embedding_as_list = embedding.tolist()

    # Prepare the tuple for upserting into the index
    # Look at the link to the pinecone docs
    # https://docs.pinecone.io/docs/python-client#upsert-vectors
    # The last parameter is a dictionary of metadata
    # that you can use to store additional information
    # about the vector. In this case, we are storing
    # the quote and the author.
    tuple_for_upsert = (
        md5_hash,
        embedding_as_list, 
        {
            "quote": the_quote + " - By " + author
        }
    )
    
    # Upsert the quote into the index
    index.upsert([tuple_for_upsert])

from sentence_transformers import SentenceTransformer
import pinecone

# User input
question = "What is the meaning of life?"

# Connecting to Pinecone
pinecone.init(
    api_key="YOUR-API-KEY", environment="YOU-REGION"
)
index = pinecone.Index("YOUR-INDEX-NAME")

# Encoding the user input
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
sentence_embedding = model.encode(question)

# Querying data from Pinecone
# Official docs: https://docs.pinecone.io/reference/query
results = index.query(
    vector=sentence_embedding.tolist(),
    top_k=10,
    include_values=False,
    include_metadata=True,
)

matches = results["matches"]
quotes = []
for match in matches:
    the_quote = match["metadata"]["quote"]
    print(the_quote + "\n")
    quotes.append(the_quote)

Below are some quotes from some wise people. Answer the question: "What is the meaning of life?" based on the below quotes. 

Make sure to follow the following rules when answering the question:
- Summarize what they are saying in your own words in a single short paragraph.
- You do not need to mention the authors names. Just summarize their ideas.
- Do NOT use any ideas other than the quotes.
- If the question cannot be answered based on the quotes, then say, "I dont know how to help you sorry."

The Quotes: 

Life has no meaning. Each of us has meaning and we bring it to life. It is a waste to be asking the question when you are the answer. - By Joseph Campbell

The meaning of life is contained in every single expression of life. It is present in the infinity of forms and phenomena that exist in all of creation. - By Michael Jackson

The meaning of life is that it stops. - By Franz Kafka

The purpose of life is to live it, to taste experience to the utmost, to reach out eagerly and without fear for newer and richer experience. - By Eleanor Roosevelt

Life is a culmination of the past, an awareness of the present, an indication of a future beyond knowledge, the quality that gives a touch of divinity to matter. - By Charles Lindbergh

Sixty-four thousand dollars for a question, I hope they are asking you the meaning of life. - By Paul Scofield

For me life is continuously being hungry. The meaning of life is not simply to exist, to survive, but to move ahead, to go up, to achieve, to conquer. - By Arnold Schwarzenegger

The aim of life is to live, and to live means to be aware, joyously, drunkenly, serenely, divinely aware. - By Henry Miller

We die. That may be the meaning of life. But we do language. That may be the measure of our lives. - By Toni Morrison

The purpose of our lives is to be happy. - By Dalai Lama

Life has no inherent meaning, but rather we create meaning by living and experiencing it. The meaning of life can be found in everything around us, in all the forms and phenomena that exist in creation. Life is transient, and it is this very transience that gives it its significance. The purpose of life is to live it fully, to experience it in its entirety, to constantly seek out new and richer experiences, and to move forward and conquer. Being aware, joyous, serene, and drunk with the beauty of existence is the ultimate goal. Happiness is the ultimate aim of our lives, and language may be the measure of our existence.

$ pip install openai

from sentence_transformers import SentenceTransformer
import pinecone
# Import OpenAI
import openai

# The question we want to answer
question = "How to not be worried about the future?"

# Connect to Pinecone
pinecone.init(
    api_key="PINE-CONE-KEY", environment="us-west4-gcp"
)
index = pinecone.Index("quotes")

# Connect to OpenApi
file = open("api_key.txt", "r")
openai.api_key = file.read()

# Embedding the question
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
sentence_embedding = model.encode(question)

# Querying data from Pinecone
results = index.query(
    vector=sentence_embedding.tolist(),
    top_k=10,
    include_values=False,
    include_metadata=True,
)

# Loop through the results and get the quotes
matches = results["matches"]
quotes = []
for match in matches:
    the_quote = match["metadata"]["quote"]
    quotes.append(the_quote)

the_prompt = '''\
Below are some quotes from some wise people. Answer the question: "{}" based on the below quotes. 

Make sure to follow the following rules when answering the question:
- Summarize what they are saying in your own words in a single short paragraph.
- You do not need to mention the authors names. Just summarize their ideas.
- Do NOT use any ideas other than the quotes.
- If the question cannot be answered based on the quotes, then say, "I dont know how to help you sorry."

The Quotes: 

{}

Answer:
'''.format(question, "\n\n".join(quotes))

# We will use the Davinci engine for this task
model_name = "text-davinci-002"

# Generate the answer using OpenAI
# The temperature parameter is used to control the randomness of predictions
# The higher the temperature, the more random the predictions
# The lower the temperature, the more expected the predictions
# So we set the temperature to 0.3 to get more expected results
# The n parameter is used to control the number of results to return
# The max_tokens parameter is used to control the length of the results
response = openai.Completion.create(
    engine=model_name,
    prompt=the_prompt,
    temperature=0.3,
    n=1,
    max_tokens=500
)


generated_text = response.choices[0].text.strip()
print(question + "\n")
print(generated_text)