Sadly no, I got no reply… I would really like to go deeper into this direction.
Do you have any AI/programming knowledge?
No, this is why we want to source out this project.
Are you willing to also pay for this project? Currently I’m looking for someone who could implement this for my company. Maybe we could share the costs and also the result ;o)
sure! just PM me. This makes totally sense.
New user so I can’t PM yet, but I would consider adding some financial support for this.
Removed by moderators, as this was advertisement of third-party services.
Removed by moderators, as this was advertisement of third-party services.
Hi @agitos,
Nice to see, that more and more people are interested in this topic. I did some prototyping in this couple of months ago, but some other duties have been coming in, so I put this project on-hold.
I already created a prototype based on FAISS and Streamlit, so the basic concept was working. The biggest challenge I was facing was, how to clean up the data (split the conversation into threads and remove the PII) and how to correctly chunk it. Then another topic is the Zammad integration (UI wise, webhook to get notified about new tickets etc.). Also interesting topic is to handle multi-languages tickets.
What programming language have you used and what framework? I was using python and langchain.
Hi @abal
I started AI development with Python/langchain and switched over to Java/langchain4j as I am used to work in the Java ecosystem. I focused on local processing and local embedding stores to keep internal data in the own domain. Although using AI cloud services may be more cost efficient, faster and on a higher quality level. Universal AI tool implementations let the user decide about the ai interfaces/service providers to use.
Regarding zammad I was thinking about additional data sources to include in the embedding store besides ticket conversations. Is this an important point?
A plugin-architecture to continuously load text data from different knowledge bases (files, sql, …) seems reasonable to me. Furthermore the optional inclusion of customer static data as ‘customer context’ as part of the LLM prompt might be helpful as well. A requirement might be to limit the LLM input data to the set of information a customer is allowed to see (permission management).
How did you attempt to solve the PII part? Maybe a pre-processing via LLM prompt “anonymize this text …” could help out?
Regarding the chunking I had best results with a bert-based tokenizer and a semantic splitting.
Did the prompt in your prototype include more than “answer to the last incoming conversation based on …” ?
Hi @agitos,
Currently I’m prototyping with Azure AI as this states a little higher level of discretion as OpenAI, but as you mentioned Langchain allows easy switch between different AI services.
Including more data sources is exactly my case. I want to include our technical documentation, knowledge base and later more resources.
Currently, I’m playing with the embedding, how to pre-process the tickets to get correct data in the vectors to get then correct answers when doing the RAG. I also played by neutralizing the vector DB to EN language so that I can use all information in a language neutral form. Then during answering, I can do the translation before vector search to EN and then translating the answer back to the original language. This seems to be working quite well.
Regarding the PII removal, I was trying out presidio_anonymizer, but it seems using AI and correct prompting produces better results.
@agitos you mentioned that you have hosted zammad, so maybe you know the answer: is it possible to add a link into every ticket? I need a permanent link, which will include the ticket ID as a parameter. I was trying to solve it by the object fields, but these are showing the link only in case if this field will contain a value.
I want to use these links to open a chat (currently streamlit) and talk about the given ticket or see its summary. This/these link(s) should be permanently shown for all tickets.
@abal : I think such a static external link with the ticket number as a parameter could be added to app/assets/javascripts/app/views/ticket_zoom/attribute_bar.jst.eco
or app/assets/javascripts/app/views/ticket_zoom.jst.eco
using the tag <%- @ticket.number %> in the link.
For non docker environments compiling the assets with
zammad run rake assets:precompile
... restart zammad ...
should show the custom link in the output.
As far as I know such customizations are overridden by zammad updates; at least when the modified view was changed ‘upstream’. So I checked if a custom zammad package like How to create your custom Zammad Package may be a more stable solution but I did not find any view extension approach in there. But good to know about such package development; the link URL may be administered via a custom package in your case.
Regarding the aspect of DE/EN translations before using RAG I see that multi-lingual embeddings and LLM processing is doing a very well job with mixed language data. So maybe it suffices to prompt the LLM text generation in the last step to output one language only without the auto-translations beforehand.
Things are different when you don’t use vector search on multi-lingual embeddings to find matching documents for RAG input: lately I experienced rather bad vector search results in my project and therefore I’m thinking of a combined approach with traditional full-text search where language consistency matters.
BTW: I just used ‘baai-bge-reranker-v2-m3’ to re-rank text from search results; it does a good job in my use-case to filter out irrelevant search results. This reduces the context/input for the LLM processing of the answer.