What is AI Data Sculpture?
last updated: 2021-06-13 first posted: some time in 2019, I don’t remember the exact date.
Simplified view of Reality
Why the term “AI sculpture” or “AI Data Sculpture” feels weird; let’s dive into that now.
-> Side kick: In my previous post titled How do you make music with Artificial Intelligence?, I give a brief introduction and suggest readings about AI. I am passing those discussions here. The type of AI these artists and I are using in their work is Applied Artificial Intelligence (autonomous software), not General Artificial Intelligence (the kind that would conquer the world).
Some prefer the term autonomous agents instead of AI. The term AI is problematic because it creates this notion that there is an intelligence related to the autonomous software, without clarifying what intelligence is. Although we may percieve some behaviours that are generated by the software as similar to behaviour of us—intelligent beings, it is arguable to call these software “intelligent.”
...(*￣０￣)ノ
Back to the term AI Data Sculpture...
(＃`Д´) AN OBJECT IS NOT AN AGENT....!!!!
Ohommm... I am calm now.
The simplified view of reality has three components: agents, objects, and the environment. Agents and objects situate in the environment. Objects are static, they do not react to the changes in the environment, they do not interact with other objects. Agents can act on the enviroment, react to the changes in the environment, and interact with other agents.
Let me explain reactivity and interactivity briefly. First, let’s give an example of reactive behaviour using house cleaning. Let’s say that you are a clean and tidy person. You (agent) desire to keep your house clean. Your house (environment) has become dirty (changes in the environment), you (agent) realize this at some point (agent’s perception). Since you (agent) intend to live in a clean house, you (agent) plan to clean your house. Your cleaning plan includes actions, such as preparing the cleaning supplies, cleaning one room at a time etc.
What would be interactivity then? Interactive behaviour happens between agents. Let’s say that you start cleaning your house. You move a piece of furniture, and there is a mouse running out. You immediately stop cleaning and get out of the house and keep running...☆ﾐ(o*･ω･)ﾉ
By jumping at you, the mouse has changed your actions that are planned previously. By moving the furniture, you made a change in the behaviour of the mouse, and it run away from you to find another hiding spot. Notice that this is different than reactivity. After you see the mouse, you don’t give a heck about the cleaning anymore, you just run away full speed. Both agents actions are affected by each other.
When we mention the term sculpture, my immediate connotation is an object. You may think that the examples of kinetic sculptures contradict the idea that a sculpture is an object. Although kinetic sculptures are moving and dynamic, they do not exhibit agent behaviours such as proactivity, reactivity, or interactivity. When the artwork exhibits reactive and interactive behaviours, then we move towards the realm of Interactive Arts, and the discussion of agent behaviours. In the case of many artworks that are labeled as AI Sculptures, the artworks are fixed-media videos that are played on a screen, and sometimes these screens are installed on a custom-shaped object; hence appearing more like an object than an agent.
A follow-up question is, how do we label these works? Let’s brainstorm. Given that these artworks use content that is generated by an Applied AI algorithm, maybe “AI generated Data” is a better fit? Still though, public may percieve “AI generated Data” as a content that is generated by an Artificial General Intelligence with intrinsic motivations, which is not the case. Joanna Bryson---an AI ethics researcher mentions, communicating a work to the public in this manner is problematic. Another issue is, if the artwork is a “Data Sculpture”, wouldn’t we expect some relationship between the properties of the data, and the form of the sculpture? Do “AI Data Sculpture” artworks have this relationship?
PS: I started writing this article sometime in 2019, and it was online, but not listed on my website until now. Somehow, the page moved its way up on the google searches of “AI Data Sculpture.” Internet has its own marvelous universe.
The terminology of agents and the concept of simplified view of reality is based on the book:
- Wooldridge, M. (2009). An Introduction to MultiAgent Systems. John Wiley & Sons.
An extended discourse on the difference of reactivity and interactivity can be found here with a focus on interactive music systems:
- Tatar K. & Pasquier P. (2019). Musical agents: A typology and state of the art towards Musical Metacreation. Journal of New Music Research, 48(1), 56–105.
- Joanna Bryson on AI ethics: http://www.cs.bath.ac.uk/~jjb/web/ai.html
An Introduction to Creative Artificial Intelligence for Music
Updated on 30-01-2020
I get this question a lot:
How do you make music with Artificial Intelligence?
This question goes as deep as the question “how do you make music?”; but still, I will try to give a general introduction to making music with Artificial Intelligence. There will be bunch of readings in the following, yet this is not a page that gives an explicit implementation tutorial.
Disclamier: This is an informal page. I may speak out my personal views, and speculate here and there. Yet, the publications are solid, formal, and peer reviewed. Well, maybe too formal for those who are not familiar with the academic language.
Open Access: Some papers are not open access, that is, you need a subscription to the journal to download the article. Most university libraries are subscribed to these journals and you can access papers through your library. Please contact your library if you can’t access any paper. If you don’t have any ties to an academic institute, you could just ask the authors to share an author copy with you. The contents of author copies are the same as the journal copies. The author copies are just not formatted in the journal style. You could easily send the authors an email, or send them a message on researchgate to ask for an author copy. Sometimes, the authors also share their papers on their personal websites. Open access is a known issue in academia, and many researchers are aware of the issue and proposing solutions. However, things are moving with an institutional speed, aka slow as hell.
Let’s do it.
Will AI conquer the world?
My computer can’t even find a file if I mistake one letter. Good luck on conquering the world. Pop-science articles on AI are detrimental to the society, and let’s stop sharing these clickbaits. There are other issues related to AI technologies that need immediate actions. I will touch on these under the section AI & Ethics at the bottom. Spoiler: I don’t think autonomous systems will conquer the world, simply because they lack instrinsic motivations. My computer will not wake up one day and start cooking some delicious food, unless I program it to do so. Yet, this doesn’t mean that humans will not make malicious autonomous software to cause real harm. Actually, this has happened already:
Documentary: Zero Days
Let’s clarify what we mean when we call a system AI system. Two main disciplines around artificial intelligence appear in the research:
Artificial General Intelligence (AGI) and Applied Artificial Intelligence (AAI)
Artificial General Intelligence (AGI) is quite philosophical and mainly focuses on human cognition, how we communicate etc. AGI systems try to solve multiple problems at once with one architecture, and they are mainly implemented as computational systems, aka software. That is asking the question if we could have a system that cooks, communicates, talks, plays music, choreographs dance etc.
Side kick: If you have spare time, here is a paper that compares various cognitive architectures in the AGI domain. This paper is not necessarily related to music. However, the CLARION architecture mentioned in this paper has been applied to music.
-> Thórisson, K., & Helgasson, H. (2012). Cognitive Architectures and Autonomy: A Comparative Review. Journal of Artificial General Intelligence, 3(2), 1–30. https://doi.org/10.2478/v10229-011-0015-3
Applied Artificial Intelligence (AAI) is autonomous systems solving one problem at a time. Almost 99% of AI systems you see online are in this category. These systems automatize one task in a smart way. AAI will be our main focus, and from now on, I refer to AAI when I mention artificial intelligence.
What is Musical Metacreation?
Making music with Artificial Intelligence =Creative Artificial Intelligence for Music = Musical Metacreation <- Well, sort of...
The following paper gives an introduction to the general idea of making music with AI, and Computational Creativity. -> Pasquier, P., Eigenfeldt, A., Bown, O., & Dubnov, S. (2017). An Introduction to Musical Metacreation. Computers in Entertainment, 14(2), 1–14. https://doi.org/10.1145/2930672
Looking from a Wider Perspective
Making music with AI is strongly connected with a broader discipline called Computational Creativity (CC).
CC asks the questions,
-> What is creativity?
-> What would be the definition and taxonomy of creativity?
I strongly recommend the writings of Margaret Boden if you are interested in these two questions. Here are some:
A must read -> Boden, M. A. (2015). Creativity and ALife. Artificial Life, 21(3), 354–365. https://doi.org/10.1162/ARTL_a_00176
Additionals ->Boden, M. A. (2010). The Turing test and artistic creativity. Kybernetes, 39(3), 409–413. https://doi.org/10.1108/03684921011036132
->Boden, M. A. (2009). Computer models of creativity. AI Magazine, 30(3), 23.
->Boden, M. A., & Edmonds, E. A. (2009). What is generative art? Digital Creativity, 20(1–2), 21–46. https://doi.org/10.1080/14626260902867915
-> How do we model human creativity?
-> How do we support creativity with autonomous systems?
-> How do we make artificially creative systems?
These are more broad questions. Pasquier et al. (2017) gives a nice introduction to these questions in the domain of music. The first paper in the next section also covers these questions applied to music.
Can I play music with Artificial Intelligence?
Yes. ︎We recently published a survey on interactive music systems based on artificial intelligence, aka musical agents. The topic goes deep, and the paper is quite heavy. Still, I strongly recommend the sections 1, 2, 3 for an introduction to the topic; and section 8 to understand where we are at in this area. If you have time, feel free delve into the whole paper.
-> Tatar, K., & Pasquier, P. (2018). Musical agents: A typology and state of the art towards Musical Metacreation. Journal of New Music Research, 47(4), 1–50. https://doi.org/10.1080/09298215.2018.1511736
You can download a free eprint copy of this paper using this link.
Can I compose music with Artificial Intelligence?
Yes. I prefer categorizing these systems as purely generative as they run offline and do not interact with other software agents and humans (excluding the user interaction with hyper-parameters). Following paper surveys these systems from a conventional music perspective:
-> Herremans, D., Chuan, C.-H., & Chew, E. (2017). A Functional Taxonomy of Music Generation Systems. ACM Computing Surveys, 50(5), 1–30. https://doi.org/10.1145/3108242
And, the following paper covers purely generative systems that apply Deep Learning:
-> Briot, J.-P., Hadjeres, G., & Pachet, F. (2017). Deep Learning Techniques for Music Generation-A Survey. ArXiv Preprint ArXiv:1709.01620.
I already mentioned so many papers! I recommend the following, find a specific musical task that you want to focus, and read related sections in those papers. From there, you can follow the citations in those sections to go deeper in your focus. Some systems share their codes as a link in the paper, you can download and check them out.
AI & Ethics
Following two papers cover Ethics in Artificial Intelligence for musical applications. They are still in my to-read list, and I am sure they will be a fun reading to cover.
-> A. Holzapfel, B. L. Sturm, and M. Coeckelbergh, “Ethical dimensions of music information retrieval technology,” Trans. Int. Soc. Music Information Retrieval, vol. 1, no. 1, pp. 44–55, 2018.
-> B. L. Sturm, M. Iglesias, O. Ben-Tal, M. Miron, and E. G ́omez, “Artificial intelligence and music: Open questions of copyright law and engineering praxis,” MDPI Arts, vol. 8, no. 3, 2019.
Before we finish, I mentioned earlier that I will touch on AI & Ethics in general. I think the most immediate action is required to ban AI weapons. Think about this way, companies could make a turret to automatically detect and shoot any enemy entering an area. This may sound futuristic, but here we are: wikipedia.org/wiki/SGR-A1
I think that no machine should make the decision to end a human life. There is almost a consensus in the academia about this:
-> futureoflife.org/lethal-autonomous-weapons-pledge/?cn-reloaded=1&cn-reloaded=1
-> www.theguardian.com/technology/2017/aug/20/elon-musk-killer-robots-experts-outright-ban-lethal-autonomous-weapons-war
We should take this matter as serious as nuclear weapons. Next time you are about to share another AI conquering world article, please share an article about why we should ban autonomous weapons instead.
There are also social effects of autonomous systems. I really like Johanna Bryson’s writings on that topic: http://www.cs.bath.ac.uk/~jjb/web/ai.htm.
Like all the topics I mentioned so far, AI & Ethics goes quite deep. IEEE started the discussion of standards for autonomous systems, and you can find a book on this topic here:
https://standards.ieee.org/industry-connections/ec/autonomous-systems.html
Last Words...
I feel like I talked about readings of hundreds of pages, without giving explict guidelines for the implementation of autonomous systems making music. Well, that is another post I guess.
PRESS
____〆(￣ー￣ )
Respire, breathing in sound and vision
The Artificial Intelligence Takeover of Music in 2019
Excalim!
Published on Dec 12, 2019
by Daryl Keatin
mentioned with Holly Herndon, YACHT, Endel, and Algorave artists.
︎
a Canadian Music MagazinePublished on Dec 12, 2019
by Daryl Keatin
mentioned with Holly Herndon, YACHT, Endel, and Algorave artists.
